{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bagging\n",
    "\n",
    "<p>Bagging is a procedure that let's use bootstrapping to reduce the variance, and ultimately performance, of a prediction. The procedure can be defined as follows:<br><br>\n",
    "We have data $D=[(X_1, Y_1),(X_2, Y_2),...,(X_N, Y_N)]$ and we want to learn:$\\:\\: E[Y|X]=\\hat{f}(X)$. Define a bootstrap sample $D^b$ as $N$ samples from $D$, sampled with replacement. Let $E^b[Y|X]=\\hat{f}^b(X)$ be the function learned from training set $D^b$.\n",
    "Our bagged prediction is then the mean of all estimates of $f^b(X)$. I.e.,\n",
    "<br><br>\n",
    "<center>$\\hat{f}_{bag}(X) = \\frac{1}{B}\\sum\\limits_{b=1}^B \\:\\hat{f}^b(X)$</center>\n",
    "<br>\n",
    "This is a relatively straightforward procedure to implement, which we can show on a simulated example.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import os\n",
    "os.chdir(\"C:/Users/kevin/Documents/GitHub/DS_course/ipython/\")\n",
    "import course_utils as bd\n",
    "import imp\n",
    "imp.reload(bd)\n",
    "\n",
    "#Generate Y, and X, where E[Y|X] is a 3rd order polynomial\n",
    "betas = [0, 2, -2.5, 1]\n",
    "n=200\n",
    "sig=0.2\n",
    "sp=20\n",
    "\n",
    "x_init = np.random.uniform(0, 1, n)\n",
    "e_init = np.random.normal(0, sig, n)\n",
    "\n",
    "dat = bd.genY(x_init, e_init, betas)\n",
    "dat = bd.makePolyFeat(dat, 6)\n",
    "\n",
    "#Plot the data vs. the real curve\n",
    "plt.plot(dat['x'], dat['y'], 'b.')\n",
    "plt.plot(dat['x'], dat[['x','x2','x3']].dot(np.array(betas[1:])), 'r.')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Now let's do bagged prediction using linear regression\n",
    "from sklearn import linear_model\n",
    "\n",
    "n_boots = 100\n",
    "\n",
    "boot_est = dict()\n",
    "x_grid = np.arange(0, 1, 0.01)\n",
    "d_grid = pd.DataFrame(x_grid, columns=['x'])\n",
    "d_grid = bd.makePolyFeat(d_grid, 6)\n",
    "\n",
    "\n",
    "fig = plt.figure()\n",
    "\n",
    "#Generate and plot each bootstrap model\n",
    "for i in range(n_boots):\n",
    "    D_b = dat.iloc[np.random.randint(0, dat.shape[0], size=dat.shape[0])]\n",
    "    regr = linear_model.LinearRegression(fit_intercept=True)\n",
    "    regr.fit(D_b.drop('y', 1), D_b['y'])\n",
    "    boot_est[i] = regr.predict(d_grid)\n",
    "    plt.plot(x_grid, boot_est[i], color='0.75')\n",
    "\n",
    "#Now aggregate the bootstrapped models for a single prediction\n",
    "bag_est = pd.DataFrame(boot_est).mean(axis=1)\n",
    "\n",
    "#For comparison, we'll also see what a single fit on the original data looks like\n",
    "regr = linear_model.LinearRegression(fit_intercept=True)\n",
    "regr.fit(dat.drop('y', 1), dat['y'])\n",
    "non_bag = regr.predict(d_grid)\n",
    "\n",
    "#Truth\n",
    "truth = d_grid[['x','x2','x3']].dot(np.array(betas[1:]))\n",
    "\n",
    "#Now Get MSE of estimates\n",
    "mse_bag = round(np.sqrt(((truth-bag_est)**2).sum()), 3)\n",
    "mse_non = round(np.sqrt(((truth-non_bag)**2).sum()), 3)\n",
    "\n",
    "\n",
    "plt.plot(x_grid, non_bag,'k-', label='Non Bagged Estimate, MSE={}'.format(mse_non))\n",
    "plt.plot(x_grid, bag_est,'r-', label='Bagged Estimate, MSE={}'.format(mse_bag))\n",
    "plt.plot(d_grid['x'], d_grid[['x','x2','x3']].dot(np.array(betas[1:])), 'b--', label='Truth')\n",
    "\n",
    "\n",
    "plt.title('Bagging Example')\n",
    "plt.legend(loc=4)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can see in the above example that variance of the bootstrapped predictions is very high, especially around the extreme values of $X$, where there is less data. The goal of Bagging is to reduce the variance of the prediction (and thus improve accuracy). The lever that controls the amount of variance reduction is the number of bootstrap samples used.\n",
    "<br><br> \n",
    "<b>When to use Bagging</b><br><br>\n",
    "According to the original <a href=\"http://statistics.berkeley.edu/sites/default/files/tech-reports/421.pdf\">Bagging paper</a> by Leo Breiman, bagging will produce a much better reduction in training error when the underlying classifier or regression model is unstable, and sensitive to minor variations of the data $D$. The above example was meant to show Bagging at work. We used a linear regression on polynomial features, and even though the underlying model was misspecified (we used degree 6 where the truth is degree 3), the bagged estimate isn't too far from the estimate without bagging. One must be careful when choosing to use bagging, because it can actually hurt performance if the underlying model is fairly stable (if you rerun the above regression example with different data sets or different bootstrapping iterations, you might find that the bagged MSE is worse).\n",
    "</p>\n",
    "\n",
    "## Random Forests\n",
    "### The Basic Algorithm\n",
    "<p>The Random Forest algorithm is probably the most well known and utilized implementation of the Bagging technique. A RF is an ensemble of Decision Trees, where both bagging and random feature selection are used to reduce the variance of the forest. The basic algorithm goes as follows:<br><br>\n",
    "Assume we have a data matrix $D=[X,Y]$ with $N$ records and $M$ features.<br><br>\n",
    "\n",
    "<i><u>Train</u></i><br>\n",
    "For each $b$ of $B$ iterations:\n",
    "<ul>\n",
    "    <li>Draw a bootstrap sample $D^b$ of size $N$ from $D$.</li>\n",
    "    <li>Sample $p$ features from $X$, where $p<<M$.</li>\n",
    "    <li>Grow a Decision Tree $T_b(X)$ on this data</li>\n",
    "</ul>\n",
    "<br>\n",
    "<i><u>Score</u></i><br>\n",
    "Take the average of all of the tree predictions, i.e.<br>\n",
    "$RF(x)=\\frac{1}{B}\\sum\\limits_{b=1}^B \\: T_b(x)$\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We'll start by building a forest using <a href=\"http://scikit-learn.org/stable/modules/classes.html#module-sklearn.ensemble\">SKlearn.ensemble</a>. Check out <a href=\"http://scikit-learn.org/stable/modules/ensemble.html\">here</a> for a brief overview of Ensemble methods in Python. We'll also compare this to a single tree. This test is close to an off-the-shelf test, in the sense that we are not doing any intelligent hyper-parameter optimization. However, we do use a few well reasoned starting parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "import pandas as pd\n",
    "import course_utils as bd\n",
    "imp.reload(bd)\n",
    "\n",
    "\n",
    "f = 'C:/Users/kevin/Documents/GitHub/DS_course/datasets/Cell2Cell_data.csv'\n",
    "dat=pd.read_csv(f, header=0, sep=',')\n",
    "train, test = bd.trainTest(dat, 0.8)\n",
    "lab = 'churndep'\n",
    "\n",
    "\n",
    "#We'll build a RF and compare to a DT\n",
    "clf_def = DecisionTreeClassifier(criterion='entropy', min_samples_leaf = 20)\n",
    "clf_def = clf_def.fit(train.drop(lab, 1), train[lab])\n",
    "dt_pred = clf_def.predict_proba(test.drop(lab,1))\n",
    "\n",
    "rf_def = RandomForestClassifier(criterion='entropy', n_estimators=100)\n",
    "rf_def = rf_def.fit(train.drop(lab, 1), train[lab])\n",
    "rf_pred = rf_def.predict_proba(test.drop(lab,1))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now compare the forest to the tree using AUC</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_curve, auc, roc_auc_score\n",
    "import course_utils as bd\n",
    "imp.reload(bd)\n",
    "\n",
    "\n",
    "bd.plotAUC(test[lab], dt_pred[:,1], 'DT')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "bd.plotAUC(test[lab], rf_pred[:,1], 'RF')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can see that with little optimization involved, the RF is already a bit better. Let's see how much better we can do with a grid search on both the DT and the RF. We have already covered DT optimization in a different module, so here we'll focus on the input parameters of the RandomForestClassifier object that let us tune the forest.<br><br>\n",
    "\n",
    "The following two are forest specific:\n",
    "<ul>\n",
    "    <li>n_estimators - the number of trees (and bootstrapped samples) to be used</li>\n",
    "    <li>max_features - the number of features that will be randomly sampled for each tree.</li>\n",
    "</ul>\n",
    "The default in RandomForestClassifier is max_features=sqrt(total_features), which is generally a good suggestion. The default for n_estimators is 10, which is probably too low. The other design parameters are specific to the individual decision trees, which are covered in the decision tree module.\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Out-of-Bag Error\n",
    "<p>Usually we would want to use cross-validation for performance tuning. For Random Forests, this becomes problematic, as cross-validation with RF's can be painfully slow. That's because each cross-validation step requires building k*n_estimators trees. One advantage of a random forest is that it performs what is called an \"out-of-bag\" error calculation. Remember that each tree is built from a bootstrap sample of the data, so that for each tree, some portion of the data is not used for that tree. The RF method computes an out-of-bag prediction for each record $[x_i, y_i]$ by averaging the prediction $f^b(x_i,y_i)$ on record $i$ for the bootstrap iterations in which record $i$ was not chosen in the bootstrap. The out-of-bag prediction can then be used to compute out-of-sample error for model selection and validation. This method should be equivalent to $N$-fold cross-validation. \n",
    "\n",
    "<br><br>\n",
    "We'll start by building a tree of oob predictions enabled and then compare the performance to a true hold out set.\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "rf_def = RandomForestClassifier(criterion='entropy', n_estimators=100, oob_score=True)\n",
    "rf_def = rf_def.fit(train.drop(lab, 1), train[lab])\n",
    "rf_pred_test = rf_def.predict_proba(test.drop(lab,1))\n",
    "rf_pred_train = rf_def.predict_proba(train.drop(lab,1))\n",
    "\n",
    "bd.plotAUC(test[lab], rf_pred_test[:,1], 'Test')\n",
    "bd.plotAUC(train[lab], rf_pred_train[:,1], 'Train')\n",
    "bd.plotAUC(train[lab], rf_def.oob_decision_function_[:,1], 'OOB')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can see two very important things here. \n",
    "<ul>\n",
    "    <li>The training AUC is 1.00. This means that the RF perfectly predicts the training set!</li>\n",
    "    <li>The OOB AUC is exactly equal to the test AUC up to 2 digits of precision.</li>\n",
    "</ul>\n",
    "Now we'll use OOB error to do model selection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "n_est = [50, 100, 200, 500, 1000]\n",
    "m_feat = [1, 3, 6, 11]\n",
    "\n",
    "aucs_oob = {}\n",
    "aucs_test = {}\n",
    "\n",
    "for m in m_feat:\n",
    "    aucs_oob[m] = []\n",
    "    aucs_test[m] = []\n",
    "    for n in n_est:\n",
    "        rf_oob = RandomForestClassifier(criterion='entropy', n_estimators=n, max_features=m, oob_score=True)\n",
    "        rf_oob = rf_oob.fit(train.drop(lab, 1), train[lab])\n",
    "        aucs_oob[m].append(roc_auc_score(train[lab], rf_oob.oob_decision_function_[:,1]))\n",
    "        aucs_test[m].append(roc_auc_score(test[lab], rf_oob.predict_proba(test.drop(lab,1))[:,1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#We'll plot in this block\n",
    "\n",
    "x = np.log2(np.array(n_est))\n",
    "for m in m_feat:\n",
    "    plt.plot(x, aucs_oob[m], label='max_feat={}'.format(m))\n",
    "    \n",
    "plt.title('OOB AUC by Max Feat and N-Estimators')\n",
    "plt.xlabel('Log2(N-Estimators)')\n",
    "plt.ylabel('OOB-AUC')\n",
    "plt.legend(loc=4, ncol=2, prop={'size':10})\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can see in the plot above that the forest gets better with more trees, but we see that this effect tapers off asymptotically. We also see somewhat of a Golidlocks phenomenon with respect to the optimal number of features. Having too few features is suboptimal (i.e., max_feat=1), probably due to high bias in the trees. Conversely, having too many features (max_feat=11) is also bad, and this is because the individual trees are too correlated with each other, which prevents the Bagging approach from reducing the overall variance (see the analysis below to understand the math behind this statement). The right number of features is in the middle. We see that choosing $3$ or $6$ perform relatively the same. $3$ happens to be closer to the good heuristic default, which is max_feat=sqrt(total feats). \n",
    "<br><br>\n",
    "Next, for the purpose of educating ourselves, we'll also look at the Test set AUC's on each of the design options above.\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEZCAYAAACq1zMoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4VEX3xz+HXkLvRZqIVAkJAlYi9i5W9KWLgNhefzbs\n6GvDhooVhChWVFRsKKgEpBogCTWEGiAESKghgZByfn/MTbIJKRuSze4m83me+2T33pm55+5m7/ee\nMzNnRFWxWCwWi6W0qeRtAywWi8VSPrECY7FYLBaPYAXGYrFYLB7BCozFYrFYPIIVGIvFYrF4BCsw\nFovFYvEIVmAsXkdEqotIpoi09LYtFRVf/w5EZLOI9PW2HZbiYQXGRxGRJBE54mwZIpLisu/2ErS7\nVETucKNcPRE5JiKz8uzP90YkIi+LyBSX9/VFZLKI7HBsjhGRV0WkfgGnLJUJWSIyxrHvxTz7b3P2\nv18a58nT9tcicty5zqzv6LoStumNG36B34GILBORoyLS1GXf1SKyoZA6L4vICZf/4yQR2V2UESLy\nlYg8kcsw1Y6qutzdC3EXEblcRDaVdrsWgxUYH0VV66hqXVWtC8QCV7vs+6oMTLgNSAauEpEGec0r\nrKKI1AAWAO2BAc41nOe0F1RQtZKZm4vNQF4RHQpsLMVzuKLA8853k/Ud/VTCNoVSEt1inrMgFDgG\nPJHP/sL4JOv/2PlsfM1DKtHnLCKVS9GWcocVGP9AyPPjF5FKIvK0iGwRkX0i8pmI1HWO1XKeAveL\nyEHHa6knIq8DZwMfO0+UrxVyzmHAJGALkNdjKkoMRgENgBtVdTOAqiao6nOq+nch9QaKyDYR2Ssi\nLzjXUlNEDovI6S7X3lpEkrOuNx9igW0icpFTvikQCMxxaaOyiHwnIntE5ICI/CUinZxj1UVknYiM\ncin7r4g8XMR1n4Rj648ikuCEeca4HDvX8QwOisguEXlTRLJ+kwucvzEFeUQicqaIzHe+570i8omI\n1HY5Hi8i/xWRNc45PhORKi7Hn3SufwcwmKJvtG8BI0SkdXE/h3xsryQi7zn/u4dEJEJEzhCR+4Cb\ngKed657pci3nOq9fFpHPHc8xSURWiUg7EXnG+Zy3ikh/l3ONFpENLp70CGd/Q+B7oIOL59lARGo4\ntu0W44G/miUkWR6PiDwlInuA90WkmYjMcT7jBBH5s6SfT3nBCoz/8ghwCXAu0BpIwwgCmBt8ZaAF\n0Ai4Fzihqg8D4cCdzhPlI/k17Nxo+wJfOtuwYtp2MfCrqp4oZr1rgLOAPsDtInKHqh4DvsXcALO4\nA/hFVY8U0I4CM8ix+z/AN0BGnnI/Yrys5kA08CmAqqY653vFEbYJQLKqvl6ci3HE4jdgkXOOK4DH\nReQCp0gacI+qNgAucK5/lHPsQoyQn1GER/Qc0BToAXQCnsxz/CbgIqAj0A/HsxORG4Cxznk7A1e6\ncUnbgM+AZ90oWxTXYES/varWd+w6qKqTgVnA/5zrvq2A+jcA7wP1gBjgb+Ao0Ax4E/jApexu4HLH\nkx4LvCciXVT1ADAQ2OrieR4Enge6A92AYCAEeNSlvXaY31dr4H7gMcz/T0PMb27CKX4m5Q4rMP7L\nGGC8qu51buT/AwY5x9KAJpibU6aqrnRu1FkU5YEMBf5V1W0YgektImcWw7ZGQHwxymfxkqomqWos\n8C45ntMMcgvMYMyNrjC+Ay4XkVqY65nhelBVM1T1c1U95vL5nS0i1ZzjEcAbwM+Ym9LQIs73lOMJ\nHXQ8AjA37+qq+rpzvs3AJzjfk6qGq+pK5/U2YBrQP0+7BX5XqrpRVcOctvcBb+dT/01VTVTV/Rix\nC3T23wJMVdVNqpqCESp3eBG41dWjLIKhzueStf3q7E8D6gJdRURUdYOqJrrZJsBfqrpQVTMx33Ud\nVX3Tef81cKaIVAdQ1V9UdYfz+m+Md3h+IW3fATyjqgdVNQF4ARjicvw48IKqpjsPI2lAS6Cds29R\nMa6jXGMFxn85Dfgt64cLrIJst38asBD4znHxXxSR4vRxDAa+AFDV7cAycryBLC+gap46VTE/NID9\nmCe54rLL5XUs5keLqi4EKolIXxHpifEG5uRTPxtVPQr8hXnaruwIRjZO2OsNMSHGQ8AGzM28kUux\nUMyT/4+qurMI219Q1Yaq2kBV2zj72gDtXW6uB4EHMU/ZiEgXEfnNCVMdBp4GGhdxHtdraCEi3zjh\ntUPAx/nU3+vyOgUIcF63BFyvKRY3+sFUNR6YQh5BEpGRLmEm14EhM5zPJWu72mlnDub/9CMgXkTe\ndR4G3MX1uo4BCXneA9R2bLtORJY7ocSDGI+usM+5ObDD5X0s0Mrl/R5VdfWGX8A8UM13QnAPFuM6\nyjVWYPyXXZgO9IYuN7baqnpAVU+o6rOq2gUTarmFHO+mqA76izA3xglO3DseE7YaDKCq6ZiQQ7s8\nVdtjfogAf2IGB1Qr5jWd5vK6jXOeLGZgniKHAF/n+YEXxGfAQ+TxXhxGYkJ5/Z0QTWdnv+tN9iNM\nuGagiAS7dQW52QlsyPMd1VPVm53jU4GVmDBRPYwXlXV+dzqeX8OEhbo61zAKN0TCIZ7cn3dbN88J\nMBET4uqetUNVp7uEmW5ypxFVfUtVgzD/X4HAA1mH3LSjSBzR+gYjiI2dcOR8Cv+c4zGfRxZtgThX\n010LO173f1W1HSYk+ZSInFM6V+DfWIHxXz4CJmZ1uIpIUxG5xnl9sfN0LJgbUDo5nsdeoEMh7Q7H\nhIW6Aj2dLRBoKCIXO2W+AZ51nqAricgVmP6g753j04ADwLcicoZjUxOnE/aiQs79mIjUFZF2mH6j\nr12OfQbcihHK/AQjP+YBl2E+q7wEYEIdB0UkABP6yUZE7gLOwHwejwCfZ4VcisEip60HxAwcqCIi\nPUQkK0xVBzisqsdEpBtwV1ZFJ2x3iMK/qzqY7/eoiLQB/q8Ytn0DjBLTsR6A8Z7cwgllvY35XE4J\nxxsNdjrPjwEngEzncFH/o26dwvlbE6iC4+GIGSwR4lJuL9BUXAZHYP7vnhWRhmIGiDxBISFZEblW\nRNo7b5Mwv7fMgspXJKzA+Af5PWVNxNxA/3bCK4uAXs6xVsBs4AiwGtMh/o1zbBIwzAkXvOLaoPMj\nGwi87Yz62udsWzA/uqww2VNABLAUEw57HrjFZcTYccyPONax7wiwGKiFE8or4Bp/BaKAf4GZqvpF\n9kHVrZhhxkmquqKwD8ulTqaq/q2qSfkcngYkAnuccy50+Rw6AK8Ag1U1VVVDMSG0VwuxPb/zpwNX\nYQZixGJuZu+TE6Z6ELjL+Xwmk1tQAZ7BhDkPZD085HP8AowQzcL0RRRpl2Pbj5hQ1z/AeuD3gsoW\n0NYbRZ3DYZjkngdzRETqAPUx/VEHMcPKt2FEC8euPs51f+nmefK11+l7ehj4BfN9X4f5P8M5HgX8\nBMQ656uP+VzXA+sw/6//YLzFguiCCY8dAcKA19QDc3b8EVEPLzjmPN2+hRGzaao6MZ8yIZgbX1Ug\nQVUvKqyuE4f/EKiBifuPc/emY/FfROQLYJ2qvuRtWywWS9F4VGCcYZoxmFj3bswQ2UGqGu1Sph6w\nBLhMVeNEpLGqJhZWV0T+AN5Q1bkiciXwaJYoWconItIRWAF0cTqaLRaLj+PpEFkfYJOqxqpqGiYE\ncH2eMncAs1Q1DrLju0XVzcSMfwfjart2wFnKGSIyEdMZ/pwVF4vFf6hSdJES0YrcQyF3YYTDlU5A\nVRGZj4lNv6OqnxVR90HgDxF5A9OZd64HbLf4CKr6GGYym8Vi8SM8LTDuUAWTn2oAZtz6UhFZWkSd\nu4EHVPVHEbkZmA5c6lkzLRaLxVIcPC0wcZj5DFm05uRw1i4g0Rl5dFxEFmKGxhZWd5iqPgCgqt+J\nyLT8Ti4iZZ0s0GKxWMoFqlriBLSe7oMJBzqKSFtn0t0gzJBAV2YD5zszq2thcmBtKKDubKdOnDjJ\n7Jy5GTEFGaCqPrU9++yzXrfBH2zyVbusTb5lU2ZmJhsTNzIjcgb3/HoPvaf0ptaLtQj+KJhxv4wj\nNCKUd5a9w52z76TFNS2o9WItOr7TkZtm3sTzYc8zO3o22w9uJzMzs9x/VsXZSguPejCqmiEi9wJz\nyRlqvEFMRllV1SmaMypsNWYy4BRVXQ+QT92s0Wd3Ae84k7SOA6M9eR0Wi8U32J+yn3/j/mV53HKz\n7VpO3ep16du6L31b9eX27rcT1CKImlVrnlR3wqoJPDX+KWL2xxC1J4qovVF8sOIDovZEcSz9GGc1\nO4uezXoS2DyQns160q1pN2pUqeGFqyw/eLwPRlV/B87Ms++jPO9fB07KVJtfXWf/EqB36VpqsVh8\niRMZJ4jaE8XyuOUs27WM5XHL2Ze8j94te9O3VV/GBo8l9PpQmgc0d7vNKpWq0LVJV7o26crtPXJW\noUhITiBqbxRRe6II2x7G28vfJmZ/DB0adKBns55ma27+Ng9ojhQrtV/FxRc6+SsUISEh3jbhJHzR\nJvBNu6xN7lFcm1SV7Ye2ZwvJ8rjlrN67mo4NO9K3VV8uancRj5//OJ0bd6ZypVNb46swm5rUbsIl\nHS7hkg6XZO9LTU9lQ+KGbG/ntSWvEbUnikpSKVtssjyezo07U7Vy3vyvJbfL3/H4TH5vIiJanq/P\nYvFXDh8/fFKoq2rlqvRtZUJd/Vr3I7hlMAHVAopurAxRVXYn7c72dqL2mi32UCxnNj7zJG+nUa1G\nRTfqg4gIWgqd/FZgLBaLR0nPTGfN3jW5xGTH4R0EtQiiX+t+RlRa96V13RIvlOk1UtJSWLdvHZF7\nIrNFZ/Xe1dSpVieXt9OzeU/OaHjGKXthZYUVGDewAmOxlC2qyq4ju7KFZFncMiLiI2hbv222d9K3\ndV+6N+1OlUrlO0KfFfbL6+3sPbqXbk275RKds5qdRd3qBa0AXvZYgXEDKzAWi2c5euIoK3avyOk7\n2bWcDM3IJSZntzybejXqFd1YBeFI6hHW7F2TS3jW7ltL09pNT/J22tdv75UBBVZg3MAKjMVSemRk\nZrA+YX22kCyPW86Wg1vo2axntpj0bdWXdvXb2VFWxSQjM4PNBzaf5O0cST2SPXw6S3S6N+1OrarF\nWfyz+FiBcQMrMBbLqROfFJ9LTFbsXkHzgObZQtK3VV96Nu9JtcrFXbjU4i77U/azeu/qbMGJ2hNF\ndGI0beq1OcnbaVWnVakJuxUYN7ACY7G4R0paCqviV2WLybJdy0hOS6ZPqz7ZYtKnVR+/HRVVnkjL\nSGPj/o1mQIGLt5ORmXGS6HRp3IXqVYq7EKsVGLewAmOxnEymZhKzPyaXmEQnRtOtabfsIcJ9W/Wl\nY8OONtTlR+w5uieX4ETtiWLrwa10bNjxJOFpWrtpoW1ZgXEDKzAWi5ml/m/cv9kd8eG7w6lfo37O\nEOFWfenVopdNi1IOOZ5+nPUJ64naE5VrCHWNKjVypcUJ1uacvuMola+5FrAC4xZWYCwVjdT0VCL2\nRGR7J8vjlpOYknhSqKtZQDNvm2rxEqrKziM7jbezJ5K63/3MHZ+u4tfrOjPs87WAFRi3sAJjKc+o\nKlsObsklJmv3raVTo065hgl3btyZSuLpxOkWv2PfPhg7FjZtghkzyOh5VvYE0NISmPI908liKUcc\nPHbwpPQqNavWzBaTm7veTHCLYGpXq+1tUy2+zqxZcM89MGIEfPUVVK+OJ3ILWA/GYvFB0jLSWL13\nda7kj7uTdhPcIjjXnJNWdVt521SLP3HgANx3H6xYAZ9+Cv365VvMhsjcwAqMxR9QVWIPx+YKdUXu\niaR9/fY5o7pa96Vrk67lPr2KxYP89huMHg033wwvvQS1Cp6saQXGDazAWHyRI6lHCI8LzxXqAujb\nui/9Whkx6d2yt0/lprL4MUeOwEMPwZ9/wvTpcNFFRVaxAuMGVmAs3iY9M511+9blSv64/dB2ejXv\nlSvU1aZeGzvnxFL6/P03jBwJl10Gb7wBdeq4Vc0KjBtYgbF4g8g9kcxcO5Mlu5awKn4Vreq0ypVe\n5axmZ53y4lQWi1ukpMD48fD99zB1Klx5ZbGqW4FxAyswlrJif8p+vlzzJaGRoSSmJDL4rMH0b9uf\nPq360KBmA2+bZ6lILFkCw4dD377wzjvQoPj/f34jMCJyBfAWUAmYpqoT8ykTAkwCqgIJqnpRUXVF\n5D5gHJAO/Kqq4/Np1wqMxWNkZGYwb+s8pkdMZ+6WuVx1xlWMCBzBgPYDfH5BKUs5JDUVnn3WjA57\n7z248cZTbsov5sGISCXgXeBiYDcQLiKzVTXapUw94D3gMlWNE5HGRdV1BOlaoIeqpmfVsVjKgk37\nNxEaGcqMqBm0rNOSEYEj+Oiaj6ynYvEeK1fCsGFw5pkQFQVNC881VlZ4esxjH2CTqsYCiMjXwPVA\ntEuZO4BZqhoHoKqJbtS9G3hFVdPz1LFYPMLRE0f5dt23TI+cTsz+GAb3GMzvg3+ne9Pu3jbNUpFJ\nS4MXX4T334e33oLbbwcfGiziaYFpBex0eb8LIxyudAKqish8IAB4R1U/K6JuJ+BCEXkJOAY8oqor\nPGC/pQKjqizasYjQyFB+iP6BC9pcwEPnPMRVZ1xl10CxeJ+1a43X0qwZREZCy5betugkfGHWVhUg\nCBgA1AaWishSN+o0UNV+InI28A3QwbNmWioKu47sYkbUDEIjQ6laqSojAkfw0j0v0TygubdNs1gg\nI8MMOX7tNXjlFTMM2Ye8Flc8LTBxQBuX962dfa7sAhJV9ThwXEQWAj2LqLsL+B5AVcNFJFNEGqnq\n/rwGTJgwIft1SEgIISEhJbkeSzklNT2V2RtnExoZyvJdy7ml6y18PvBz+rTqY+enWHyHTZuM11Kj\nBoSHQ7t2pdJsWFgYYWFhpdKWKx4dRSYilYGNmI76eOBf4HZV3eBSpjMwGbgCqA4sB25z6uVbV0TG\nAC1V9VkR6QTMU9W2+ZzfjiKzFEpEfATTI6bz1dqv6Nm8JyMCR3Bjlxs9vua5xVIsMjPNyLDnn4dn\nnjGJKit5LkO2X4wiU9UMEbkXmEvOUOMsgVBVneKMCvsDWA1kAFNUdT1AfnWdpqcD00VkDZAKDPXk\ndVjKF4kpiXy55kumR0zn0PFDDA8cTvhd4bRv0N7bplksJxMba7IeHzsGixdDp07etsht7ERLS4Ug\nPTOduVvmEhoZyrwt87im0zWMCBzBRe0vsmulWHwTVZM7bPx4eOQRk0+sctnMr/ILD8Zi8TYx+2MI\njQhlxuoZtK7bmpGBI5l67VTq16jvbdMsloLZvRvuugvi42H+fOjun8PhrcBYyh1JqUl8s+4bQiND\n2XxgM4PPGszcwXPp1rSbt02zWApH1SwA9uCDcPfd8OSTUNV/89ZZgbGUC1SVf3b8w/SI6czeOJv+\nbfvzyLmPcNUZV9nEkhb/ICHBiMqGDWbtluBgb1tUYqzAWPyanYd3Zs9ZqVGlBiMCRzDxkok0C2jm\nbdMsFvf54QcYNw6GDoXPPzfDkMsBVmAsfsfx9OPMjp7N9MjphMeFc1u32/jqpq/o3bK3nbNi8S8O\nHoT774dly+C77+C887xtUaliBcbiF6gqq+JXERoZytdrv6ZXi16MCBzBj7f9SM2qNb1tnsVSfH7/\n3XTkDxxoUr3Uru1ti0odKzAWnyYhOYEv1nxBaGQoR1KPMCJwBCtHr6Rt/ZPm1Vos/kFSEjz8MPzx\nh0mtP2CAty3yGFZgLD5HemY6f2z+g+mR0/lr619ce+a1vHX5W/Rv19/OWbH4N2FhZtLkxRfD6tVQ\nt663LfIodqKlxWeITowmNCKUz1Z/Rtv6bRkZOJJbu91KvRr1vG2axVIyUlLgiSdMP8tHH8HVV3vb\nokKxEy0t5YIjqUf4Zt03TI+YzrZD2xhy1hD+GvoXXZp08bZpp8ahQxAaCr/8YibHnX++6bj1wVTq\nljJi2TKToLJ3b+O1NGzobYvKDOvBWMqcTM1kYexCQiNDmR09mwHtBzAicARXdLzCf+esrFljkhHO\nnAlXXgm33QYbN5rcUYsXm1DIeecZwTn/fOjSxaPJCi0+QGoqTJhgHjjefRduvtnbFrmN9WAsfseO\nwzv4NPJTPon6hFpVazEycCSvXfoaTWv7xvKuxSY9HWbPNjePjRth7FhYvx5atMhdTtUcX7TIbK+/\nDvv3w7nn5ojO2WeXm7kPFiAiwsxp6djRLGHcrGLOy7IejMWjHEs7xo/RPxIaGcrK+JUM6jaIEb1G\nENwi2H/nrOzbBx9/DB98YNbjuPdeM9S0WjFWudyzB5YsyRGd9evhrLNyQmrnnQeNG3vsEiweIi0N\nXn7ZPHS88QYMHuyzi4EVRml5MFZgLKWOqrIyfiXTI6Yzc91MglsEM7LXSG7ofAM1qvjxU3p4uLlx\n/PQT3HSTEZbAwNJpOzkZ/v3XhNMWLYKlS40nlBVSO+888zTshzerCsP69cZradzYPIC0bu1ti04Z\nKzBuYAWmbElITuDz1Z8zPXI6KWkpDO85nGGBw2hTr03RlX2V1FT49luYPNl4LvfcY5ao9XRHbUaG\nWXN90aIc0UlNzd2P06uXXydCLDdkZMCkSTBxIrz4opk86ecPAlZg3MAKjOdJz0xnzqY5TI+czvxt\n87m+8/WMDBzJBW0v8O85K7t2wYcfwtSp0LMn3HcfXHVVma3HkS87duSIzeLFsGWLGZmUJTrnnAP1\n7JDuMmXzZhg+HKpUMWu3dOjgbYtKBSswbmAFxnNsSNhAaKSZs9KhQQdGBI7g1m63Ure6H08cU4WF\nC00Y7K+/TPx83Djo3NnbluXP4cMmlJYlOitWmBtcVkjt/POhjR97j75MZqbpg5swAZ56yjyAlKNR\ngVZg3MAKTOmSkpbC56s/JzQylNhDsQztOZThgcPp3NhHb8DukpwMX3xhhCUtzfStDBnif7Os09LM\n6CXXsFr16jlic9550KOHd72w8sCOHSZMevSoSfVy5pnetqjUsQLjBlZgSg9VZeDMgaSkpfBA3we4\nvOPlVKnk56Pct2yB9983N4nzzzfCcvHFfh8/z0bVXGPWSLXFi81Kif365fTj9OlTLpMsegRV+OQT\nePRRs3zxww+b0Fg5xG8ERkSuAN4CKgHTVHViPmVCgElAVSBBVS9yp66IPAS8BjRW1QP5tGsFppT4\nJPITJi2bxL+j/qV6lereNufUycyEuXONt7J8uXkSvftuM9y4IpCYmHt4dFQUdOuW28tp3tzbVvoe\n8fEwerTpm/v0UzOkvBzjFwIjIpWAGOBiYDcQDgxS1WiXMvWAJcBlqhonIo1VNbGouiLSGvgYOBMI\ntgLjOWIPxdJ7am/+GvoXZzXz0x/W4cPm6fO998wT+333we23Q80Knur/2DHTd5MVUluyxIyQc+3H\n6dy5/Hh1xUXVZGd44AEYM8b0txRnvpOf4i8z+fsAm1Q1FkBEvgauB6JdytwBzFLVOABVTXSz7iTg\nEeAnD19DhSZTMxk+eziPnPuIf4rLunVGVL7+Gi6/3IjMOedU3BtmXmrWhAsuMBsYD2/DBiM2//xj\nJg0eOWKyDmSJTu/epm+nvJOYaAZ5rF1rcsudfba3LfI7PD3soRWw0+X9LmefK52AhiIyX0TCRWRI\nUXVF5Dpgp6qu8YzZlizeWf4OaRlpPHTOQ942xX3S080StAMGwKWXmjQd69bBV1+ZG6UVl4KpVMmE\nzMaMgRkzYOtWk6BxyBCIizNP8o0aGbEZP97ceA+cFDzwf2bPNmGwtm1h1SorLqeIL/RQVQGCgAFA\nbWCpiCwtqLCI1ASeAC513V1Q+QkTJmS/DgkJISQkpGTWViDWJ6znxX9eZNmdy6hcyQ9GHiUk5KRw\nadPGdNrfeGOFCGl4lJYt4ZZbzAZm9NTy5cbLefttuOMOOO203JNA27f3TyE/dMiI6OLF8M035loq\nAGFhYYSFhZV6u57ug+kHTFDVK5z34wF17awXkceAGqr6nPP+Y2AOEJdfXeBX4E8gBSMsrZ2yfVR1\nX57z2z6YUyQtI41zpp3D6ODRjA4e7W1zCmfFCtNpP3u2EZR77oGgIG9bVXFITzdeTlY/zqJFJtTm\n2o8TGOj7I67mzoVRo+Daa82s/IAAb1vkNfylk78ysBHTUR8P/AvcrqobXMp0BiYDVwDVgeXAbU69\nQus69bcBQap6MJ/zW4E5RZ6d/yzhu8P59Y5ffTMpZWqqWbxp8mSTOHLcOLjzThO+sXgXVYiNzT0f\nJzbWhJmyRKdfP9+ZZ3T0KDzyCPz6K0ybZsKqFRy/6ORX1QwRuReYS85Q4w0iMsYc1imqGi0ifwCr\ngQxgiqquB8ivbn6noZAQmaX4/Bv3Lx+u/JDIMZG+Jy5xcWZFwKlTzYJejz8O11xjJw/6EiJm2He7\ndiYbAsDBgybrwKJFJl/XypXQqVPu4dHeSA65cKFZwrh/f7Omj021U6rYiZaWXKSkpRD0URDPX/Q8\nt3a71dvmGFTNjendd2HePBPzv+ces2iXxT85ccJ0nrtOAq1dO3c/Trdunku/cuwYPPmkGYL80Ufm\nIcWSjV+EyLyNFZji88CcB0g8lsgXN37hbVPMOuZZKVxSU02n/dChvhNasZQeqhATk7sfJyHBDCnP\nEp0+fUpn3tLy5WYJ4169zP+WDauehBUYN7ACUzz+2voXw2cPZ/XY1TSo2cB7hmzdalK4fPKJublk\npXApR8kELW6wd29O1oHFi00Iq0eP3IuyNS3GaqipqfD882ak4eTJcKuPeOg+iBUYN7AC4z6Hjh+i\n54c9mXrtVC47/bKyNyAz04S/3n0Xli0zcfG77zbDXS0WMB5teHiO4CxZYuY4ufbjdOqU//DoqCjj\n/bZrZ0JiNh1OoViBcQMrMO4z9Ieh1KlWh/eufq9sT3z4sMnt9N57JvyRlcKlVq2ytcPif2RkmAm0\nrmvkpKTkeDfnn28mS06aZObrvPaaERlfG7jig1iBcQMrMO4xa/0sxv81nsgxkdSuVkaZddevN6Ly\n1Vdw2WUmDHbeefbHbykZO3caockSnXXrICTEDD8+7TRvW+c3WIFxAyswRbPn6B4CPwzkh9t+4JzT\nzvHsydIESDyaAAAgAElEQVTTTWqRyZONwIwebVKStGzp2fNaKi7HjkGNGvbBpZhYgXEDKzCFo6pc\n//X19GjagxcvftFzJ0pMNE+Q778PrVqZMNhNN9kULhaLj+IXEy0tvk1oZCg7j+zku1u/88wJVq0y\nnfY//AADB8L330NwsGfOZbFYfA4rMBWUbQe38difjzF/2HyqVS5FT+LECZPC5d13zaz7ceNg0yZo\n3Lj0zmGxWPwCKzAVkKw1Xh477zG6N+1eOo3u3m2Gf06ZYmZgP/qoSRpoU7hYLBUWO3OtAjJp6SRU\nlQf7PViyhrJSuAwaZPKCJSbCX3/Bn3/CDTdYcbFYKjjWg6lgrN23llcWv8LyUctPfY2XlBQzvPjd\ndyE52Qwx/ugjmyjQYrHkwgpMBeJExgmG/jCUly9+mQ4NOhS/gd274a23IDTUpFufOBEuucSmcLFY\nLPli7wwViP8t+B8t67Tkzl53Fr/yzJlm0ai0NJMs8OefzQRJKy4Wi6UArAdTQVi2axlTV00lcmwx\n13g5eNCkxl+1yizIZNcmt1gsbmIfPysAKWkpDP1hKO9d9R7NA4qR5O/PP00up0aNjMBYcbFYLMXA\nzuSvANz3230cSj3EZwM/c6/CsWNmpcjvvjMz8C+/3LMGWiyWMiU9M5MNKSmsOnqUVUlJrExKom2N\nGnzRtStgZ/Jb3GTelnnM3jib1Xevdq/CqlVmmdvu3WH1amjY0LMGWiwWj5Kamcna5GRWJSVlC8ra\n5GROq16doDp1CAoI4PrGjQkMCCj1c1uBKcccPHaQO3+6k2nXTaN+jfqFF05Ph1dfNanN33rLLEts\nEwRaLH5FSkYGUUePZgvJqqNHiU5JoWPNmgQFBBBUpw53NG1KYEAAdap4/vbv8RCZiFwBvIXp75mm\nqhPzKRMCTAKqAgmqelFhdUXkVeBaIBXYAoxQ1SP5tFuhQ2SDvx9MgxoNmHzV5MILbtli1smoUcOs\nImnTmlssPs+R9HQiXYRkZVIS244fp0utWtmeSVCdOvSoXZtaxZz07BfZlEWkEhADXAzsBsKBQaoa\n7VKmHrAEuExV40SksaomFlZXRC4B/lbVTBF5BVBVfTyf81dYgfl23bc8Nf8pIsZEUKtqAYt3qZrl\nY594Ap58Eu6/3w47tlh8kP1paUS4hLhWHT1KXGoqZwUEZAtJUEAA3WrXplop/Ib9pQ+mD7BJVWMB\nRORr4Hog2qXMHcAsVY0DUNXEouqq6p8u9ZcBN3n0KvyM+KR47p1zLz8N+qlgcdm7F0aNgl27ICzM\n5A+zWCxeZ++JE7m8klVJSRxITyfQEZOrGjXiqbZt6VyrFlV8/IHQ0wLTCtjp8n4XRjhc6QRUFZH5\nQADwjqp+5mZdgJHA16VmsZ+jqtz1812MDhpN39Z98y80ezaMHWvWvZ81y67LYrF4AVVlV2pqLq9k\nVVISxzIzs72SW5s04ZUOHehYsyaV/LBP1Bc6+asAQcAAoDawVESWulNRRJ4E0lT1y4LKTJgwIft1\nSEgIISEhJbHV55kWMY34o/F83//7kw8mJcF//wvz58O335o1yy0Wi8dRVbYeP55LSFYdPYoAwXXq\nEBwQwIjmzZncsSNta9Qo3mToUiAsLIywsLBSb9fTfTD9gAmqeoXzfjymv2SiS5nHgBqq+pzz/mNg\nDhBXWF0RGQ7cBQxQ1dQCzl+h+mC2HtxK34/7EjYsjG5N84S8Fi0yHfkDBpiRYnXqeMdIi6Wck6HK\nJpc5Jll/61Spkqu/JKhOHVpWq1bmYuIO/tIHEw50FJG2QDwwCLg9T5nZwGQRqQxUB/oCbwIbC6rr\njC57BLiwIHGpaGRkZjDsx2E8fv7jucXlxAl49lkzOuyjj+C667xmo8VS3kjLmrDoIiRRyck0rVo1\nW0gea9OGXgEBNK2AoWiPCoyqZojIvcBccoYabxCRMeawTnFGhf0BrAYygCmquh4gv7pO05OBasA8\nR/2Xqeo4T16Lr/Pm0jepLJX5b7//5uxct85MmjztNIiMhGbNvGegxeLnuE5YXOkIytrkZNq4TFgc\n6ExYbFC1qrfN9QlsqphywJq9axgwYwDhd4XTrn47yMyEt9+Gl16Cl1+GO++0kyYtlmKQ34TFjXkm\nLAYFBNCzjCYsljX+EiKzeJgTGScY8sMQJl4y0YjLjh0wfDikpsKyZXD66d420WLxaY6kpxORp79k\n2/HjdHUmLJ5dpw5jWrakR+3a1LSrtBYL68H4OU/+9SRr9q1h9m0/Il99BQ8+aLZHH7VLFlsseShq\nwmKw45l0LaUJi/6Kx2fyi0gw0FhV/8iz/wpgr6pGlPTknqa8C8zSnUsZOHMgq28No+kjz8LatfDZ\nZxAU5G3TLBavkKHKgbQ0Ep0tIS2N9cnJ2YJyID2dXnlGcp1Zs6bPT1gsa8pCYP4CRqnqtjz722E6\n3C8u6ck9TXkWmOQTyQR+FMgnNW7nvOemwy23mD6XmjW9bZrFUiqoKofT07PFIr8tIc/7w+np1KtS\nhSZVq9K4alUaVa1KZ5fcXKf76YTFsqYs+mDq5hUXAFXdLiJNSnpiS8l44uf/8v7vlTlvzSfw6adw\nsc/rvaUCo6okZ2QUKhZ5t/3p6dSqVInGjljk3U6vWfOkfQ2qVLHeiA9RmMA0KORYAQmuLGXB0u/f\n4b5xn9Dmohsgago0KOyrslhKn+MZGewvxLtIOHHipH2VRLI9i7xbz4CAk/Y1qlq1QveDlAcKC5FN\nAeJV9dk8+58B2qjqqDKwr0SUuxBZejopzz1NyluvEv/i4/S4/wVvW2QpB6RnZhYqFvltqZmZBXoW\nBW3FTRlfUlTNShQHD+Z/7FT3+VP94pRt1Ah69jSvyyJE9hAwXURigKwO/UBgDTCipCe2FJNNm2DI\nEDYfi+W794fz/BArLpaTyVTlUB6xyNtPkXdLSk+nYQGicFr16vTKx7uoU7myz6U4ycyE9ethwQJY\nuNBslStDixb5l8/PfHf3+VN9d8v265cjMKVFkcOURaQTkJV7ZJ2qxpSuCZ6jXHgwqibFy9NPs2rU\nNfynxRJW3l3IGi+WcoOqklTMfosDaWnUqVKlWJ5F/SpVqOxjYuEOGRlmVe8FC8z2zz9Qrx707w8X\nXmj+tmtn5xifCh73YETkLJe3WwAFkkp6QksxiI83s/D37WPf77O4cv4t/HLTL1Zc/JRjjlgU5VG4\nbtUL6eRuW6PGSfsaVqlC1XLab5GWBqtWGc9kwQJYvBiaNzdicsstMHkytG7tbSstrhTWB/NPPrsb\nOn9vV9XVHrOqlPBrD+b772HcOBg9Gn3qKa7+9gb6tOrDhJAJ3rbMUgTbjx3j+dhY4lJTc4lFumqB\nndz5bY2qVq3QM8dTUyE8PCfktXSp8UiyPJQLL7Tp9TyF15ZMFpG+wKuq2r+kJ/c0fikwhw/DAw+Y\nx7MZM+Ccc5iycgpTVk5h6Z1LqVrZJtHzZb5PSGBsTAz3tGpF3zp1cglGbR/st/AlUlJg+fKckFd4\nOJx5phGU/v3N8kWNGnnbyoqB13KRqepyEbGLiXiCBQtg2DC44gqIiICAALYc2MKTfz/JwuELrbj4\nMMczMnhoyxbmHDjALz160KduXW+b5PMkJcGSJTkeSmQk9OhhxOSRR+C880yfisV/KbbA2EmWHiA1\nFZ5+Gj7/HKZOhauvBswaL0N/HMqTFzxJlyZdvGykpSA2pqRw27p1dKpVi4jevalXDrPrlgaHDpl1\n77I8lHXrIDjYCMqECXDOOVC7trettJQmhXXyT8J07LvSELgAM4TZUhqsXm3WbOnYEaKioEmOfr++\n5HWqVa7G/X3v96KBlsL4dM8eHt6yhRfbt+euFi1sCMyFxMSc4cILFsDmzdC3r+k7ee0187pGDW9b\nafEkhXXy35lnlwL7geWqusfThpUGPt0Hk5EBb74Jr75qfm3DhuUaT7l672ounnExK+5aQdv6bb1o\nqCU/jqanc8+mTYQnJTGza1d6BAR42ySvEx+fIyYLFsCuXXDuuTmd8r17QwVc1NEv8XgfjKpOK+DE\nLUXkQVWdVNKTV1i2bzeCogr//gvt2+c6nJqeypAfhvDapa9ZcfFBoo4e5bZ16zivXj3Cg4OpXUFH\neu3YkdN/smCB8VguuMCIyciREBgINlpYsXHr6xeRhsDNwO1Ae+BHTxpVblE1I8Meftj0Yj70UL5r\ntkwIm0CHBh0Y1nOYF4y0FISq8sHu3Ty7fTtvdezIfyrQGFlV2Lo1xztZuBCSk3MmNN57r+mgL6dT\ncCynSGF9MLWBG4A7MDP5ZwOdVLVVGdlWvkhMhDFjICYG/vyzwJwMi3cs5pOoT4gaG2Xj+T7EwbQ0\nRm3cyLbjx1nSqxdn1Crfk11VITo6xztZuNDsywp3jR8PnTvbWfKWwinseWMfMBp4Heigqg8AJ4p7\nAhG5QkSiRSRGRB4roEyIiESIyFoRmV9UXRFpICJzRWSjiPwhIr49mPG334ygdOhgBvcXIC5HTxxl\n2I/D+PDqD2lau2kZG2kpiGWHD9NrxQpaV6/O0qCgcikumZlmvMnkyXDzzWYC45VXmmHEl1wCYWGm\nT+XLL2HsWOjSxYqLpWgK6+R/GBgEVAa+AmYC81W1g9uNi1QCYoCLgd1AODBIVaNdytQDlgCXqWqc\niDRW1cTC6orIRGC/qr7qCE8DVR2fz/m928mfnGzCYXPmwCefQEhIocXH/jKW1IxUQq8PLRPzLIWT\nqcrrO3fyxs6dTDnzTK5v3NjbJpUa6elm3kmWh/LPP9C4ce5Z8m1t91+FpSw6+V8HXheRMzB9L3OA\nliLyEPCDqm51o/0+wCZVjXWM/hq4Hoh2KXMHMEtV45zzJrpR93ogK5PAp0AYcJLAeJXly2HIEDO4\nPyqqyBljczbN4ffNvxM1NqqMDLQUxr4TJxi6YQNJGRmEBwfTxs/H06alwYoVOeGuxYtN3q7+/eGO\nO+CDD6BlS29baSlvFNnJr6qbgOeB50UkECM2f2E6+4uiFbDT5f0ujHC40gmo6oTGAoB3VPWzIuo2\nU9W9jn17RMR34klpafDCC/Dhh/DeeybeUAT7U/Yz6udRfD7wc+rV8O1oX0Xg74MHGbJhA8ObN2dC\nu3Z+mTzy+HHzjJPloSxfbqZaXXghjBplFkFtYqdMWzxMcQcRNlfVx4B8+1JKYEMQMACoDSwVkaXF\nbKPAONiECROyX4eEhBBSRJiqRGzcaCZNNmli4g8FLUSRh3t+u4dbu97KRe0v8pxtliJJz8zkudhY\npsXH82nnzlzasGHRlXyE5GSTDDLLQ1m5Erp2NR7KAw+YPF524VNLQYSFhREWFlbq7RYr2aWIrFLV\noGKU7wdMUNUrnPfjAVXViS5lHgNqqOpzzvuPMeG4uILqisgGIERV94pIc0zf0Em5VMqsD0bVeCsT\nJsD//md6Qd3sAf167dc8v+B5Vo5eSc2qNT1rp6VAdh0/zh0bNlCjUiVmdO5M8+rVvW1SoRw5YsJc\nWcOG16wx806yhg2fey7UsRkDLaeIt5JdFveE4UBHEWkLxGMGDdyep8xsYLKIVAaqA32BN4GNhdT9\nCRgOTASGOW14h927YcQIk2hpyRLo1MntqnFH4njg9wf47Y7frLh4kV8SExm1cSMPtG7NY23aUMkH\nh0cdOGA64rNCXtHRcPbZRkxefNGsRlgOB7dZ/JwiBUZEqqhquvN2XD77CkRVM0TkXmAuZkj0NFXd\nICJjzGGd4owK+wNYDWQAU1R1vXOek+o6TU8EvhGRkUAscGsxrrn0+PZbM8PsnnvgiSeKNW1ZVbnz\npzu55+x7CG4Z7EEjLQVxIjOTx7Zu5fuEBGZ17855PpS6d+9eIyhZHsr27Wa8yIUXwltvGXHxcSfL\nYnFryeSTwmLFDZV5C4+FyA4dMsISHm4yIJ99drGb+CD8A0IjQ1k8crFNw+8Fthw7xm3r1tG6enWm\nd+5Mw6re/Q7i4nKnXYmPN/0mWcOGg4LAyyZaKhBlsWRyU6AFUFNEepATHqsLVFxn/O+/YfhwuO46\ns2bLKcQlNu3fxNPzn2bRyEVWXLzA13v3ct/mzTzTti33tmpV5hkTVCE2NnfalUOHTB6v/v1Nwoez\nzso3i5DF4lcUNtFyBDASCAQiyBGYJCBUVb8tEwtLQKl6MMePmzDYN9/Axx+bRcFOgfTMdC4MvZBB\n3QfZNPxlTEpGBg9s3syCQ4f4umtXgsqoF1wVNm3KnWn4xImclRovvNCM+PLD0dCWckqZLZksIreq\n6jclPZE3KDWBiYgwkya7dDHzW0qwbuvL/7zMn9v+ZN6QeVQSe0cpK9YlJ3PbunUEBgTwQadO1PFw\nmt8NG2D+/BxRqVIlR1D694czzrCpViy+S1mOImsqInVV9YiIfIiZs/K4qv5V0pP7PBkZZq2WN980\n23/+U6K7QuSeSCYtm8SK0SusuJQRqsq0+Hge37aNVzt0YHjz5h4LiZ04AbNmwTvvmLxdl15qHN2X\nX4Z27aygWCoe7gjMaFV9V0Quw/TJ3AVMB8r30KetW2HoULNC0ooV0KZNiZrLWuPljcveoE29krVl\ncY8j6emMiYlhbXIyCwMD6eKh9Xj37oUpU4xze+aZ8NhjcO21tg/FYnHnMTorxnQVMENVo9ys55+o\nwrRpZj3XG280qfVLKC4Az8x/hk6NOjH4rMGlYKSlKFYmJRG0YgX1Klfm36Agj4jLihVm3bjOnY3H\n8vvvZgzIDTdYcbFYwD0PJkpEfsPkDHtCRAIoJDWLX7NvH4webSYdzJ8P3buXSrP/xP7DZ6s/s2u8\nlAGqytu7dvHSjh28d8YZ3NK0dNPUpaXlhMHi4sxo9UmTwI+yylgsZYY7AjMCEw7brKopItIYuNOz\nZnmBn38240OHDYOZM0ttFltSapJZ4+WaD2lS22YX9CT709IYER3NnhMnWBYURIeapZcdYd8+Ewb7\n4AOTrOGRR0wYzC4JbLEUjDvZlDNEpANwKfAiUJPyFCJThbvvhrlzjbBccEGpNv/Q3Ie4qN1FXHfm\ndaXariU3/xw6xH82bOC2pk35rls3qpXSmN9Vq4y3Mnu2SYw9Z46Zo2KxWIrGnWHK7wJVgQtVtYuI\nNAT+UNXiT18vY9wepvztt3D55VC3bqme/9eYX7l3zr1EjY2ibvXSbdtiyFDl5dhY3o2LY3rnzlxV\ngiHkWaSlwQ8/GGHZuRPGjTMp7kuhaYvFLyjLYcrnqmqQiEQAqOoBEalW0hP7FLfcUupNJqYkMvqX\n0Xx545dWXDxEfGoqgzdsIEOVlb1706qEYc2EhJwwWMeO8H//ZxI22DCYxXJquBNHSHOWL1YAEWkE\nZHrUKj9HVbn717sZ1G0Q/dv1L7qCpdj8ceAAwStXckG9evwVGFgicYmIMAmxO3WCbdvg11/NGvQ3\n3mjFxWIpCYXlIsvKmPweMAtoIiLPYTIXP1dG9vklX639ivUJ6/ls4GfeNqXckZaZydPbtvHFvn18\n2aULIae4ilZaGvz4owmDbd9uEmJv2mTWpbdYLKVDYbnIsjMmi0g34BJMPrI/VXVt2Zl46pTZgmMu\n7Dqyi6CPgvh98O8EtfD5hNN+Rezx49y+fj0NqlThk86daVKt+JHaxESYOhXefx/at4f77zfzVqyn\nYrHkUBZ9MNmNq+o6YF1JT1beydRMRswewf1977fiUsp8n5DA2JgYHmvThgdbty72omCRkTB5Mnz/\nvQl9/fyzWQHSYrF4jsIEpomI/F9BB1X1TQ/Y49d8EP4BR1KPMP788d42pdxwPCODh7ds4bcDB/i5\nRw/6FmOkX3q6GV78zjsm88+4cTYMZrGUJYUJTGUggOIvk1whidkfw7Nhz7J45GKqVLLxltIgJiWF\n29av54yaNVkVHEx9N1fcSkw0Kyq8/z60bZsTBrMLdlksZUthd8J4VX2+zCzxY9Iz0xn6w1AmhEzg\nzMZnetuccsFne/bwf1u28EL79oxu0cKtFDtRUSYMNmuWEZQffzQrQVosFu/gVh+MpXAmLppInep1\nGHf2OG+b4vccTU/n3k2bWJ6UxN89e9IjIKDQ8unp8NNPJgy2ebNJyrBxI5RyCjKLxXIKFDYP5uLS\nOIGIXCEi0SISIyKP5XO8v4gcEpFVzvaUy7EHRGSNs93vsr+niCwVkQgR+VdEepeGradCRHwEby9/\nm9DrQ+0aLyVk9dGj9F65kkoirAgOLlRcDhyAV1+F0083S/WMG2fmsDz5pBUXi8VXKNCDUdUDJW3c\nmaD5LkasdgPhIjJbVaPzFF2oqtflqdsNk1SzN5AO/C4iv6jqVuBV4FlVnSsiVwKvAReV1N7icjz9\nOIN/GMykyyfRum7rsj59uUFV+XD3bp7Zvp1Jp5/O4ObNCyy7erUJg333HVx/vRkVFly+VyayWPwW\nT/dG9wE2qWosgIh8DVwP5BWY/MJxXYDlqprq1F0A3Ai8jskkUM8pVx+IK33Ti+apv5+ia5Ou3NHj\nDm+cvlxwKC2NURs3svX4cRb36kWnWrVOKpORkRMGi4mxYTCLxV/wtMC0Ana6vN+FEZ28nCMikRih\neERV1wNrgRdEpAGQilnwLNwp/yDwh4i8gRGncz1kf4Es2L6AL9d8yeq7V9s1Xk6R5UeOMGj9eq5t\n1Igvunalep4MyAcOmLXf3nsPWrY0o8FuusmOBrNY/AVfGE+7EmjjrDVzJfAj0ElVo0VkIjAPOApE\nABlOnbuBB1T1RxG5GbOE86X5NT5hwoTs1yEhIYSEhJTY4COpRxg+ezhTrp1C41p2UkVxyVTljZ07\neX3nTj7q1IkbmuReJ2ftWhMG++Ybk2zyu++gt9d62SyW8k9YWBhhYWGl3m6R6fpL1LhIP2CCql7h\nvB8PqKpOLKTONiA4bx+QiLwI7FTVD0XkkKrWdzl2WFXr5dOWR1LFjPppFIIw9bqppd52eSfhxAmG\nRkdzJD2dr7p2pU2NGoAJg/38swmDRUebMNjo0dCsmZcNtlgqIGWZrr8khAMdRaQtEA8MAm53LSAi\nzVR1r/O6D0b0Djjvm6hqgoi0AQYCfZ1qcSLSX1UXiMjFQIyHryObnzf+zN/b/iZqbFRZnbLcMP/g\nQYZs2MDQ5s15rl07qlaqxMGDOWGw5s1zwmCnkGbMYrH4GB4VGGc1zHuBuZgh0dNUdYOIjDGHdQpw\ns4jcDaQBx4DbXJqY5SxwlgaMU9UkZ/9dwDsiUhk4Doz25HVkkZCcwJhfxjDz5pnUqV6nLE5ZLkjP\nzOR/sbFMjY/n086dubRhQ9atM2GwmTPhmmvM3z759c5ZLBa/xaMhMm9TmiEyVeXmb2/m9Aan8+ql\nr5ZKmxWBXceP858NG6hWqRKfdOrMynnVeecdWLcOxo6FMWOM52KxWHwHfwmRlRs+X/05Mftj+OLG\nL7xtit/w6/793BkdzV2NW1Pv1zZcMFBo2hTuu88sImrDYBZL+cYKjBvsPLyT/5v7f8wbMo8aVWp4\n2xyf50RmJo9v3cpXuxM4Z1533n23HldfDV99BX37Fl3fYrGUD6zAFEHWGi8P9nuQwOZ2AZGi2HT0\nGFcvX8/BLdWo9Gpveg6uyvvroUULb1tmsVjKGiswRfDev++RnJbMo+c96m1TfJpDh+D+H/bxRaNN\ntPq7LW/2bsWt64Tq1b1tmcVi8Ra2k78QohOjOX/6+Sy9cylnNDqjFC0rP2zYAJM+yODTgM1U63uI\nd5t0Zeg5dbDJDSwW/8V28nuYrDVenr/oeSsuecjMhDlzzKTIlYnJVHpuHVe3CODTnsHUsYvbWywW\nB3s3KICX/3mZhjUbcnfvu71tis9w9KiZFDl5MtSrr/R6Yg+rmm5l4ukdGN68uc3JZrFYcmEFJh9W\n7l7J5H8nEzEmwt40HVasgEGDoFcv+HBGOqH1Y1ienMyCroF0rV3b2+ZZLBYfxApMHo6lHWPID0N4\n+4q3aVW3lbfN8TqZmfDWW/DKKyadS4crkhi0fj0DKtfn36Agalau7G0TLRaLj2IFJg9P/v0kPZr1\nYFD3Qd42xeskJMDw4bB/PyxfDn9V382Vq7fx7hlncKtdjMVisRSBXePXhfnb5jNz3Uzev+r9Ch8a\nCwsz4bAePWDhQiWUbbyyYweLe/Wy4mKxWNzCejAOR1KPMGL2CKZeO5VGtRp52xyvkZ4O//sfTJkC\nn3wCAy7NZExMDGuTk1kSFERTm9/FYrG4iRUYh//+/l8uP/1yrjrjKm+b4jV27YL//MesGLlqFdRp\nks51a9dTGZgfGEht299isViKgQ2RAbOjZ7MgdgFvXP6Gt03xGj//bFaNvPxy+OMPqNToBCGRkZxW\nvTo/du9uxcVisRSbCu/BqCovLXqJGTfMIKBagLfNKXNSU2H8ePj+e5g1C847D2JSUrhy9WqGNW/O\n023bVvj+KIvFcmrYVDHAiYwTVKtc8foWNm0yc1vatDETKBs2hGWHD3PD2rW82KEDd9oMlRZLhaS0\nUsXYEBlUSHH58ks491wYOdJ4Lw0bwk+JiVy3di3TO3e24mKxWEpMhQ+RVTSSk82CX4sXw7x5EOis\nQPDR7t08t307v/XoQe+6db1rpMViKRdYD6YCERVlOvIzM2HlSiMuqspTW7fy+s6d/NOrlxUXi8VS\nanhcYETkChGJFpEYEXksn+P9ReSQiKxytqdcjj0gImuc7f489e4TkQ3OsVc8fR3+jCq8/z5ccgk8\n8YSZ3xIQAGmZmYyIjmbewYMs6dWL02vW9LapFoulHOHREJmIVALeBS4GdgPhIjJbVaPzFF2oqtfl\nqdsNuBPoDaQDv4vIL6q6VURCgGuBHqqaLiKNPXkd/szBgzBqFGzdasJinTqZ/Unp6dyybh1VK1Xi\nbzvHxWKxeABPezB9gE2qGquqacDXwPX5lMtvtEIXYLmqpqpqBrAAuNE5djfwiqqmA6hqYumb7v8s\nWWLSvbRuDcuW5YjLntRUQiIjaVujBj9062bFxWKxeARPC0wrYKfL+13OvrycIyKRIvKriHR19q0F\nLl2Pz0QAABuqSURBVBCRBiJSC7gKOM051gm4UESWich8EentqQvwRzIz4eWXYeBAePtts2UtXbwx\nJYVzIyIY2LgxH3bqRJVKthvOYrF4Bl8YRbYSaKOqKSJyJfAj0ElVo0VkIjAPOApEABlOnSpAA1Xt\nJyJnA98AHfJrfMKECdmvQ0JCCAkJ8dR1+AR79sCQIXD8uFnD5bTTco4tPXyYgWvX8nKHDoyww5At\nFotDWFgYYWFhpd6uRydaikg/YIKqXuG8Hw+oqk4spM42IFhVD+TZ/yKwU1U/FJE5mBDZAufYZqCv\nqu7PU8etiZblhblzTXr9O++EZ58F19WLf0xI4K6YGGZ07syVjSpuMk+LxVI0pTXR0tMeTDjQUUTa\nAvHAIOB21wIi0kxV9zqv+2BE74DzvomqJohIG2Ag0M+p9gMwAFggIp2AqnnFpSKRlgbPPAMzZsDn\nn8OAAbmPfxAXx/9iY5lj57hYLJYyxKMCo6oZInIvMBfT3zNNVTeIyBhzWKcAN4vI3UAacAy4zaWJ\nWSLS0Dk2TlWPOPtDgekisgZIBYZ68jp8me3b4fbboX59iIgA16VaVJUnt23ju4QEFvXqRQc7DNli\nsZQhNheZHzNrFtx9Nzz6KPzf/4Frf/2JzEzu2riRjSkp/NKjB43tOi4Wi8VN/CVEZvEAx47BQw/B\n77/DL79Anz65jyelp3PTunXUdOa41LLDkC0WixewY1T9jA0boF8/SEw0IbG84hKfmkr/yEg61KjB\nrG7drLhYLBavYQXGT1CF0FC48EK4916YORPq1ctdJjo5mXMjIri5SRM+sHNcLBaLl7EhMj8gKQnG\njoXISJg/H7p3P7nM4sOHuWntWl7p0IHhdo6LxWLxAewjro+zciUEBUHt2hAenr+4/JCQwMC1a/m0\nSxcrLhaLxWewHoyPompSvLz0EkyeDLfdln+59+LieCk2ljlnnUVwnTpla6TFYrEUghUYHyQxEUaM\ngL17TZLKDvkkwVFVnti2je+dOS7t7RwXi8XiY9gQmY+xYIHJgNy5MyxalL+4nMjMZGh0NAsOHWKx\nFReLxeKjWA/GR8jIgBdegA8/hOnT4cor8y93xJnjUrtSJf7s2dMOQ7ZYLD6LFRgfIC4O/vMfMxN/\n5Upo2TL/crtTU7lq9WrOrVePyWecQWUp8URbi8Vi8Rg2ROZlfv0VgoPNcsbz5hUsLhuSkzl31Spu\na9qU96y4WCwWP8B6MF7ixAkYPx6++w6+/RYuuKDgsosOHeLmdet49fTTGdq8edkZ6YO0a9eO2NhY\nb5thsZQb2rZty/bt2z3SthUYL7B5MwwaBK1amXQvhS3P8n1CAmNjYvi8Sxcua9iw7Iz8//bOPDyq\n6mzgvzeKRkow8KloZRdZwhImgcimYakf4IJaXKAKgtalWkARQb/io7X4tC5VFFu/R7FIK1UrstVS\nRfhAZFEDAZOwBjEsssuWYAJZ3u+Pe5MOwyQzCZmZZPL+nuc+c+65Z3nve+7MO+fcc95TQ9mxYwfR\n7MDUMMKNhHA0xIbIwsx770HPnnD33TBvXsXG5fXduxmTnc2nXbqYcTEMo9ZhPZgwceIEjB0LX3wB\nn37qrM4vjxJVnty+nfmHDrHS46GlTUM2DKMWYj2YMJCZCd27O+9dSl2/lMepkhJGbtrEimPHWJmU\nZMbFMIxaixmYEKLqrGvp1w8mTYK//Q0q8uZyrKiI6zIyOFFSwuLERP6rXr3wCWvUOE6dOsW1115L\nUlISH374YaXzz58/n82bNwed/vDhw/Tv35+4uDjGjh1b6fqM8LcZQEZGBr169aJTp04kJiZy6tSp\nStcbKmyILEQcPQr33QfZ2bByJbRrV3H6PSdPMjgjgz4XXshrNg3ZANLT0xER0tPTq5R/3rx53HDD\nDbRv3z6o9LGxsUyZMoWsrCyysrKqVGddJ9xtVlxczIgRI5g1axadOnXiyJEj1KtJf0xVNaQHMAjY\nDGwFJvm5ngocBdLdY7LXtXFApnuM9ZP3MaAEaFxO3RoJVq9WbdlS9eGHVfPzA6ffkJenLVat0t/n\n5GhJSUnoBazFRKpNvcnJydH27dvrqFGjtG3btnrnnXfq4sWLtXfv3tq2bVtNS0vTr7/+Wnv27KlJ\nSUnau3dv3bp1q6qqvvLKK3rPPfeoqmpGRoZ26tRJ8/08JAcOHNA2bdpofHy8ejwe3b59u65du1ZT\nU1O1W7duOmjQIN23b5+qqr711lvavXt37dq1q956662an5+vq1at0saNG2vr1q3L8gfLO++8o2PG\njKkGTdUcorXNFi5cqCNGjDgr3fj7TrlxZ//7Xx2FlFu4MwS3DWgB1APWA+190qQCC/zk7QhkAOcD\n5wCLgNZe15sCnwDf1RQDU1ys+vzzqhdfrDpnTnB5lh85opesWKF/3bs3tMJFCRW1qTMoWbWjMuTk\n5Gi9evV0w4YNqqqanJys9957r6qqzp8/X2+++WbNzc3V4uJiVVVdvHixDh06VFVVS0pKNDU1VefO\nnavdunXT1atXl1vPsmXL9MYbb1RV1cLCQu3Vq5ceOnRIVVU/+OCDsh+9w4cPl+WZPHmyvv7666qq\nOmrUKP3oo4/Krr344ovq8XjOOMaNG3davWE1MGFqtGhts6lTp+qIESN04MCBmpycrC+88EKl9KIa\nWgMT6iGyFCBbVXcAiMj7wE04PRpv/I0HdQC+UtWTbt7lwM+Bl9zrrwCPAwtCIHel2b8fRo6EvDxn\n35YWLQLnmX3gAA9lZzOrQweutWnIZ42GcXlMq1atSEhIAKBjx44MGDAAgM6dO7Njxw6OHj3KyJEj\nyc7ORkQoKioCnDUHM2bMoEuXLjz44IP06NEjqPq2bNlCVlYW1157LapKSUkJP3XdPmRkZPDUU09x\n9OhRTpw4wcCBA/2WMWHCBCZMmHC2t169hLHRorHNioqKWLlyJWvWrCE2NpYBAwbQrVs3+vXrF7Re\nQkmoDczlwC6v8904RseXniKyHvgeeFxVNwJZwBQRaQScBK4D0gBEZAiwS1UzQ7lIKFgWL3bWtYwa\nBb/9LZwbhFZf272bF3buZFGXLnS1fVxqHeeff35ZOCYmpuw8JiaGwsJCnnrqKfr378+cOXPYsWPH\naV/4rVu3EhcXx549e4KuT1Xp1KkTK1euPOPa6NGjWbBgAZ06dWLmzJl8/vnnfst46aWXmDVr1hnx\nqampTJ06NWhZaivR2GZNmzblmmuuoVGjRgBcd911pKen1xgDUxNmka0FmqtqV+B1YB6Aqm4Gngc+\nAxYC64BiEbkA+B/gaa8yImJliorgN79xjMvMmfDcc4GNS4kqE7/9ljf27GGFx2PGpZaiAf55Hz9+\nnMsvvxyAGTNmlMUfO3aMcePGsXz5cn744Qc++uijoOpr164dBw8e5MsvvwScf64bN24EIC8vj0sv\nvZTCwsLTfozi4uI4fvx42fmECRNYt27dGYc/4xLo/moj0dhmAwcOJDMzk4KCAoqKivj888/Lemk1\ngVD3YL4HmnudN3XjylDVPK/wv0XkzyLSWFUPq+oMYAaAiDyH0xu6AmgJfCNO96UpsFZEUlT1gK8A\nzzzzTFm4b9++9O3bt1pubOdOGD4cGjSA9HRo0iRwnpMlJYzevJmdBQWs9HhoXJNmexiVwrvn7NuL\nFhEmTpzIyJEjmTJlCtdff33ZtfHjxzNmzBjatGnD9OnT6d+/P6mpqVx00UUV1levXj1mz57NmDFj\nOHbsGMXFxTzyyCMkJCTw7LPPkpKSwiWXXMJVV11Fbm4uAMOGDeO+++5j2rRpzJ49m1atWgW8r1at\nWpGbm8upU6eYP38+ixYtCnpGU00nGtssPj6e8ePH061bN2JiYrj++usZXN5eHxWwbNkyli1bVul8\ngZBQ/lMRkXOALcAAYC/wNTBcVTd5pWmiqvvdcArwD1Vt6Z5frKoHRaQ5zgv9Hqp63KeO74AkVT3i\np34Nxf3NnQsPPgiPPQYTJjhu9gNxrKiIW7KyaHTuubzboQMX2D4uVUJEovLftWFECn/fKTfurEeG\nQtqDUdViEfk1zgywGOBtVd0kIg84l/VN4FYR+RVQCOQD3rvPfyQijd1rD/kal9JqCNMQWUGBY1QW\nLoT58yHId318765xSY2PZ2qbNrbGxTCMOkFIezCRpjp7MJs3wx13wJVXwvTpEB8fXL4NJ05wXUYG\nD19+OY83axZSz6V1gWjswbzzzju8+uqrpz0bvXv3Ztq0aRGUyqiIaGqzUPZgzMAEQNV5gT9hgvMS\n//77IVgbsfzoUW7bsIGX27ThzmBe0hgBiUYDYxiRpNYOkdV2cnPhoYccB5VLl0LnzsHn/fDAAR7O\nzua9hAQGuFMIDcMw6hI1YZpyjSQ93fF6fP75zsLJyhiXqbt2Mf7bb/ksMdGMi2EYdRbrwfigCtOm\nwe9+B6+95kxFDpbSNS4LDx9mhcdDi9jY0AlqGIZRwzED48UPP8Do0bB3L3z5JVxxRfB5T5aUMGrz\nZnafPMkKW+NiGIZhQ2SlfPEFeDzOLLGVKytnXI4WFjIoI4PCkhI+69LFjItRLYR7b5G0tDQ8Hk/Z\nMW/evErXWdepSXv4TJ48mebNm9OwYcNKy1Fd1PkejCpMmQJ/+hO8/TZ4LeANit0FBQzOzKR/fDwv\n2xoXoxoJ994inTt3Zu3atcTExLBv3z4SExMZMmQIMcGsJDaAmrWHz5AhQxgzZgxXXnlllWSpDur8\nkyMC9es7M8Uqa1yy8vLotW4ddzdpYgso6xA7duygQ4cOjB49mnbt2nHXXXexZMkS+vTpQ7t27Viz\nZg1paWn06tWL5ORk+vTpQ3Z2NgBTp07l3nvvBSAzM5POnTtTUFBwRh0HDx5kxIgRpKWlkZSUxHff\nfUd6ejp9+/ale/fuDB48mP379wMwffp0UlJS8Hg83HbbbRQUFLB69WoWLFjAxIkTy/IHIjY2tsyY\n5OfnR5VhidY2q1+/Pr169TrNkWcpKSkpNIn08ojq8PlfUw9CuB/MMncfl1nuBkJGeKioTXmGKh+V\nIVr3FlFV/eqrr7Rjx44aFxen8+bNq5ReqgJLl1b5qAzR3GaqFe/hExcXV6Fu/H2nqCX7wUQlHxw4\nwJjsbN5PSKC/TUOuMejTtrfI2e4Hk5KSQlZWFlu2bGHkyJEMHjyY8847LygZq4JWk/PZYIjWNqvJ\nmIGpJC/v2sUru3ezODGRLg0aRFocI0JE494i3rRr144GDRqQlZVFUlJS0HLWZKK9zWoi0TPIGmJK\nVBm/bRtv793LSo/HjEsdR6Nwb5GcnByKi4sB553Fli1baNmyZVDy1Qaisc2Cub9A9x1KzMAEwcmS\nEoZv3Mia3FxWeDw0twWUdZ5g9hZ54oknSE5OpqSkpOya794iTz75JIcOHQpYX+neIpMmTaJr1654\nPB5Wr14NULa3yNVXX02HDh3K8gwbNowXX3yR5OTkoF4Yr1ixgsTERJKSkhg6dChvvPEGjaNoK+9o\nbDNwhv4ee+wxZs6cSfPmzcumOU+aNIlmzZqRn59P8+bNefbZZ4MqrzoxZ5cBOFJYyC1ZWVxy3nn8\ntX17Ym0fl4hizi4No3oJpbNL68FUwK6CAq5etw5PXBzvJySYcTEMw6gE9pK/HDLz8rguM5NHmzZl\nfLNmkRbHiGKiaW+RuoK1WXDYEJkflh45wh0bN/JamzYMi/RCJeM0bIjMMKoX2w8mjLy/fz9jt23j\ng4QE+tkaF8MwjCpjBsZFVXl5925e3b2bJYmJdLZpyIZhGGdFyA2MiAwCpuJMKHhbVZ/3uZ4KzAe2\nu1FzVHWKe20c8Es3frqqvurGvwDcCJwEvgVGq+pxqkjpGpfFR46w0uOhmU1DrrG0aNHijCmmhmFU\nnRYtWoSs7JDOIhORGOB1YCDQERguIv7chC5X1ST3KDUuHYF7gW5AV+AGEWntpl8EdFTVrkA28GRV\nZVRVhm/cyPq8PFaEwbgsW7YspOVXhZooE/iXKycnJ6L+7ZYuXRpxH3smU/TIVBPkysnJCdl3ONTT\nlFOAbFXdoaqFwPvATX7S+ftL2gH4SlVPqmox8DnwcwBVXayqpSuhvgSaVlVAEeGuJk34NDGR+DDs\n41ITf8xrokxQM+UymYLDZAqemipXdRBqA3M5sMvrfLcb50tPEVkvIv8SkQQ3Lgu4WkQaiUh94DrA\n33zhe4B/n42QN150EedHkWtywzCMmkBNeMm/Fmiuqj+KyGBgHtBWVTeLyPPAZ0AesA4o9s4oIr8B\nClX17+EW2jAMw6iYkK6DEZEewDOqOsg9fwJnn4HnK8jzHZCsqod94p8Ddqnq/7rno4D7gP6qerKc\nsmzBhGEYRhXQWrAOJg1oIyItgL3AMGC4dwIRaaKq+91wCo7RO+yeX6yqB0WkOXAL0MONHwQ8DlxT\nnnGB6lGQYRiGUTVCamBUtVhEfo0z66t0mvImEXnAuaxvAreKyK+AQiAfuMOriI9EpLF77SH9z1Tk\nacB5wGfulNUvVfWhUN6LYRiGUTmi2lWMYRiGETmiYuqUiOSIyDcisk5EvvZzPVVEjopIuntMDoNM\nF4rIhyKySUQ2iMhVftK8JiLZ7gy6rpGWKdx6EpG2bpulu5/HRGSsn3Th1lNAuSL0TD0qIlkikiEi\ns0TkjL2MI6CrCmWKkJ7GiUime5zxPLlpwq2nCmUKl55E5G0R2S8iGV5xjURkkYhsEZFPReTCcvIO\nEpHNIrJVRCYFVWGkFxlVx4HjBaBRBddTgQVhlukdHA8D4AxFNvS5Phj4lxu+CmeYL9IyhV1PXnXH\nAHuAZpHWU5ByhVVXwE/d5/w89/wDYGQkdRWkTOHWU0cgAzgfOAdneL51hPUUjExh0RPQB2fheoZX\n3PPARDc8CfiDn3wxwDagBVAPWA+0D1RfVPRgcBZqBrqXsL3wF5GGwNWqOgNAVYv0TFc2NwF/da9/\nBVwoIiFz3RykTBBGPfnwM+BbVd3lEx9WPVVCLgi/rs4BfiIi5wL1cQyfN5HQVSCZILx68l2gvRx3\ngbYX4dZTMDJBGPSkqiuAIz7RNwEz3fBM4GY/WYNdNH8a0WJgFOeFf5qI3FdOGn+LOUNFK+CQiMxw\nu7tvisgFPml8F6F+j/9FqOGUCcKrJ2/uAN7zEx9uPflSnlwQRl2p6h7gj8BOHB0cVdXFPsnCqqsg\nZYLwPlPBLNAO9zMV7KLxSH33LlF3Jq+q7gMu8ZMm2EXzpxEtBqa3qibhNNzDItLH53rpYs6uOL7R\n5oVYnnOBJOBPrlw/Ak+EuM5ABCNTuPUEgIjUA4YAH4ajvmAJIFdYdSUi8Tj/GFvgDE01EJFfhLLO\napIprHpS1c04Qz6fAQvxs0A73AQpU0S+e+VQbTO/osLAqOpe9/MgMBenO+d9PU9Vf3TD/wbqiTP9\nOVTsxlkUusY9n43z4+7N95z+L6apGxcxmSKgp1IGA2vd9vMl3Hryply5IqCrnwHbVfWwO8wyB+jl\nkybcugooUySeKVWdoardVLUvcBTY6pMk7M9UIJki+N0D2F86RCgilwIH/KT5HmjudR6Uzmq9gRGR\n+iLSwA3/BPhvnC6pd5omXuHTFnOGAre7uUtE2rpRA4CNPskWACNdmXrgDC/sj6RM4daTF8Mpfxgq\nrHoKVq4I6Gon0ENEYkVEcNpvk0+acOsqoEyReKZE5GL3s3SBtq8rqbA/U4FkCrOehNPf9ywARrnh\nu3G2T/GlbNG8O1NwmJuvYkI9ayHUB867hfU43c5M4Ak3/gHgfjf8MI7RWQesAq4Kg1yJbqOsx/ln\nd6G3TG6a13FmZnwDJEVapgjpqT5wEIjziouonoKRK0K6ehrnBzwDZ0ZgvUjrKpBMEdLTcq86+9aE\nZyqQTOHSE45h24Ozl9ZOYDTQCFgMbMGZ4Rbvpr0M+Ngr7yA3TTbu72ygwxZaGoZhGCGh1g+RGYZh\nGDUTMzCGYRhGSDADYxiGYYQEMzCGYRhGSDADYxiGYYQEMzCGYRhGSDADY9RaRCS3GspIFJFVrhv1\n9SJyu8/1D0WkpRvOEZEPva4NFZEZ5ZS71HVtXury/x8BZBjsdX6jiEw823tzyxonIrHVVNbDIjK6\nOsoy6gZmYIzaTHUs4joBjFDVzjiuYaa6nqdxHQ7GqGqOV33JItI+SBmGq2qSqnpU9fYK0nXF8aPn\nFKj6T1V9oQr34o9HcBaNBo2IlPe78BdgzFlLZNQZzMAYUYXrymKJ2xv5TESauvGtRWS1OBvT/a60\n96Oq21T1Wze8F8cP08VucXdyptuMPwLBbgZ1xvdLRG5ze0vrRGSZ61DzWeB2t7dzm4jcLSLT3PQz\nROTPruzbxNmY6m0R2Sgif/Eq988i8rVb9tNu3BgcJ5RLRWSJGzdcnM3BMkTkD175c0XkJRFZh+P+\n5ffibEq3XkRecPWTD3wnIt2CvH+jrhNqFwl22BGqAzjuJ24BcJcbHg3MdcP/BG53ww+UkzcF2OB1\nvgzo6HW+Hcf4bABaA0OBv5Qj21IcFyrp7vG8G58BXOaGG7qfdwOveeUtOwdmAH93w0OAY0CCe74G\n6OKGS917xLh1d/KSuZEbvgzYATR20y0BhrjXSoChbrgxsNlLnoZe4f8BHo1029tROw7rwRjRRk/+\n46Dyb0Bvr/jZbtjX+SEichnOJlSjvKIvw/FH5k0x8CLOD20gfqHOEFmSqpZuMbsCmCkiv8TZQiEY\n/ul+ZgL7VLXUSekGoKUbHiYia3F8WSW4B5zu2LA7sFQd78clwCzgGq/7muOGjwH5IjJdRG4B8r1k\nOYDTKzKMgJiBMaKNYN7LnLZzoIjEAR8DT6pqmtelH4FYP/nexflhLnP5LiKfuENcb5ZXD4CqPgT8\nxs27VkQaBSHvSfezxCtcen6uOwnhMaCfqibi7DlS3ov98nZNzFdVdWUsxunNzQZuAD7xShfL6QbH\nMMrFDIxRm/H3Y7kKx80+wF3AF254NXCrGx5WVoDzDmQeMFNV5/qUtQlo41uBqhYBrwCPesUNcnsq\n91ckn4i0VtU0VX0apzfQDMgFGpZ3k75F+IlrCOQBua7b98Fe1457lf01cI2INBaRc3D0tMy3XHfb\ni3hV/QQYD3TxKq8tPtthGEZ5mIExajMXiMhOEdnlfj6CM8tptIisx3lJP85N+ygw3o2/AmcYCOB2\noA8wyn3xni4ipT+oC4F+XvV5947extmPvqIe07te05QXuXEvlr5kB1apagbOO5OE0pf8PmX4lq++\nYbeM9TgG8V2cYbhS3gI+EZEl6myH+ySOUVkHrFHVj/2UGwd8LCLf4LiZf9TrWm+cnRkNIyDmrt+o\nE4jIBerMgkJE7gCGqeotAfLEAv+HsyV3nf+iiEhXnBf8d0daFqN2YAbGqBOISB+cTaYEOALco6rb\ng8h3LbBJVXeHWMQaj4gMALJVdWekZTFqB2ZgDMMwjJBg72AMwzCMkGAGxjAMwwgJZmAMwzCMkGAG\nxjAMwwgJZmAMwzCMkGAGxjAMwwgJ/w/wVdBcMXDCHgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x288000d0f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.log2(np.array(n_est))\n",
    "for m in m_feat:\n",
    "    plt.plot(x, aucs_test[m], label='max_feat={}'.format(m))\n",
    "    \n",
    "plt.title('Test AUC by Max Feat and N-Estimators')\n",
    "plt.xlabel('Log2(N-Estimators)')\n",
    "plt.ylabel('Test-AUC')\n",
    "plt.legend(loc=4, ncol=2, prop={'size':10})\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can see similar patterns in the true holdout evaluation as we did in the OOB evaluation. We also tend to see a few more things. For one thing, having more trees actually hurts the performance for some levels of max_feat. We also see that the best choice for max_feat was indeed $3$.\n",
    "\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Importance\n",
    "<p>Much like with Decision Trees, the Random Forest Classifier has a built in mechanism for evaluating feature importance. Quoting the sklearn documentation:<br><br> \n",
    "\n",
    "<i>Features used at the top of the tree contribute to the final prediction of a larger fraction of the input samples. The <b>expected fraction of the samples</b> they contribute to can thus be used as an estimate of the <b>relative importance of the features</b>.</i>\n",
    "<br><br>\n",
    "The above computation is made for each feature in each tree and then averaged over all trees in the forest. The Random Forest Classifier returns an attribute with an importance score for each feature, and these scores sum to $1$ across all features.\n",
    "<br><br>\n",
    "We'll train a RF on the best options found above and then look at the feature importances.\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "\n",
    "rf_best = RandomForestClassifier(criterion='entropy', n_estimators=500)\n",
    "rf_best = rf_best.fit(train.drop(lab, 1), train[lab])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x288018ea90>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEoCAYAAABcllZ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXmYHUXVh99fCAkkJIQQyJAACZtoEJQthH0QwbAGQSDs\nhlUUAgIC6oeETbZPFERUFBBBZZVFFgGVYRUMSgAFBEEia5A1kV1yvj9OXabnfrP0zO2ee2dy3ufp\n597uru5T3V1dp+qcU9UyM4IgCIKgwoB6ZyAIgiBoLEIxBEEQBG0IxRAEQRC0IRRDEARB0IZQDEEQ\nBEEbQjEEQRAEbQjFEAQNiqSPSXpQ0puSDql3fhoNSf+U9Jl656M/EoqhjyHpGUlvS5oraV76barx\nnJtKeraoPOaUeZGkE3tTZkdIOl7Sz+udj3Y4GviDmS1uZuf2tnBJLZLeSYrpDUkzJR0jaVDa/8NM\nGXxP0vvp/1xJN/Z2fqvyflHK01xJr0i6RdKqmf37SPpv1Xt0Tj3z3EiEYuh7GLCNmQ03s2Hp96Ua\nz6l03p4dLC1Uo/y60eB5Hwf8raOdksp+fw34spktDiwDHAlMBW4CMLODK2UQ+DZwWSqPw81sm5Lz\nlofTU97GAi8AF1Ttv7fqPZre+1lsTEIx9E3U7kZpkqR7JL2eTBCbZvZ9UdKjqWX0D0kHpu1D8Bd9\nTLYHUt2ir+5VpG780ZIeAv4jaYCkZSRdJellSU9JOjTXxUjjJM1PefyXpFclHSRpHUkPSXpN0vcz\n6feRdLek76eW7KNZk0LKx3XpPE9I2j+z73hJV0q6RNIbwJeAbwC7put/sLP7lb0Xko6QNEfS85K+\nmNm/iKTvpN7d65LulDS4q2dUdU9+D2wG/CDlYeX0TM6TdKOkeUCzpOGSfp7u+T8lfbOd+3RWkvcP\nSeun7f+S9JKkvbt6PABm9o6Z3QlsD6wvaeuunms71zRC0m9SXl9N/8dm9t8u6cSU57mSfitpZGb/\nXume/lvSN/LKNbP3gCuAT3U3zwsqoRj6CZLGADcAJ5rZEsBRwNWSlkxJ5gBbpxbUNOC7kj5tZm8D\nWwEv5OiBVPcqpqZjR6R9vwEexFuXmwOHSdqiG5cxEVgZ2BX4Hl5hfwb4JLCLpI0zadcDngSWBGYA\nv5Y0Iu27HPgX0ATsDHxbUnPm2O2BK8xsBPBTvLV7ebr+NVOadu9X5hxNwDBgDLA/XoEvnvZ9B1gT\nmASMxE1C83M8o48ws82Bu4CvpGfyj7RrN+AkMxsG3AOcm/IxHmgG9pY0reqezkr5+BVwGbAOsBKw\nF3BuahzkwsyeBR4ANu4qbTsMAC4ElgOWB95O+c+yG7APsBQwGL9HSJoAnAfsgd/zJfGeQJdIGgrs\njpeXIAehGPom16ZW9GuSfp227QncaGa3AJjZ7/EXeOu0frOZPZP+3wXcSs9e7ixnm9kLqUW2LjDK\nzE4xsw+TrJ/iyiMPhleY75vZ74C3gF+Z2atm9gJeSa6ZST/HzM5Jsq4A/g5sI2lZYH3gGDP7wMwe\nSvnItoz/aGa/gY9ak/8/M13fr/fxCvpDM7sZ+A+wqiThimS6mb1kzn1m9gFdPKOcXGdm96X/H+BK\n9Fgze9vMZuNKaa9M+n+a2c/NJ0W7HFgWOCHdm9vSdazcDfngZpmRXaaqwsxeM7NrzOw9M3sLOBXY\npCrZRWb2VKaVX1HGOwG/MbN70r08jq7Nn1+T9BowF9iAtmUAvOfzWupNvSZpYnevqb8SiqFvMsXM\nRqZlx7RtHN6qriiM14EN8dY7kraS9MfUhX8db+mPqjEfz2X+jwPGVsn/OrB0N873cub/O3irPbu+\nWGb9+apjZ+MtyTHAa6knlN2XbV126WjPcb9eNbP5mfW3U/5G4S3dp9s5bafPKCfZvI8CBuK9owrV\n11p9DzGzV6q2Ze9rHsYCr3XzGCQtKunHyRz0BnAHMCIp0wrZ3mrlnoI/14+uPT3fV7sQeaaZjcTv\n+zvAqlX7/5jeoSXS75+6e039lVAMfZP2fAzPAj/PKIwlkmnkDHkUyVXAGcBSyYxxc+Y87bW83gKy\nJob2Kq/scc8CT1fJX9zMtuvuxeWk2oywPN6SfQEYmcwH2X1ZRVJ9vW3Wc9yvzngFeBc31VTT4TPK\ncd728voK3msYl9k2jv+vNAtD0nLA2sCdPTj8SGAVYN1kxqv0FvLc1xdxE1QlH0Nwc1KXmNlzwOHA\nORVfT9A5oRj6D5cC20naUu4IXiQ5SccAg9LyipnNl7QVsGXm2DnAkpKGZ7bNAraWtIQ8HPawLuT/\nCZgnd0gvImkhSatJWidn/vNUDlmWlnSopIGSdgY+jptpngPuBU6VNFjSGsB+wCWdnGsOMD7Tcu3q\nfnVIMtlcBJwld4IPSA7nhen8GXWb1GO5AjhF0mKSxgFf7eJau3uf/SBv7W8KXAvcl8xn3WUY3nKf\nm5zKM7px7FXAtpI2SPfyRLpxLck8+TxwUDdkLrCEYuh7tGtXTRXiFNxh+2/cpHAUMMDM/gNMB65M\nNtepwHWZY/+OOyafTiaOJrxyeRh4Bvgt7rTsMB+pktoWtwn/EzcL/QQYTj46bcW3s34/3vp8BTgJ\n2MnM3kj7dgNWwHsPVwPHmdntnci+Eq9kXpX0QLpfh9HB/cqR/6OAR4CZuLnjNPw5dPiMcpyzvXXw\n5/o2brq6E7jUzC7Kmc+OzpnlXElv4iaes/B7tVUXx3TE9/Be6Cu48r4pb17M7FHgK3g5fQG/r891\nlL6Dc/0v7ndYuBt5XiCRlfyhHkmT8QIxALjAzE6v2r89/mLPx7vFXzWze9K+Z4A3K/vMLJxDAZL2\nAfYzs2rHZRAEBTCwzJPLB+Cci4cuvgDMlHSdmT2eSfY7M7s+pV8d7xp/Iu2bDzSb2etl5jMIgiBo\npWxT0kTgSTObnULMLsO70h9RFT2yGK4MKqgX8hgEQRBkKLvSHUvb8LrnaGdQiqQdJD2GD5DaN7PL\ngNvkc7QcUGpOgz6DmV0cZqQgKI9STUl5MbNr8UFbGwEnA5XRshua2YuSlsIVxGNmdnf18ZLKdZQE\nQRD0Q8ys3ciusnsMz+Mx5BWWpZMY61Tpr1iZH8XMXky//wauwU1THR3bK8vxxx/fa7JCXsgLeQuW\nvN5cOqNsxTATWFk+SdogPOzv+mwCSStl/q8FDDKz1yQNkbRY2j4UjyP/a8n5DYIgWOAp1ZRkZh/K\nPzByK63hqo9JOsh32/nATvIZHt/HB7/skg4fDVyTzEQDgV+Y2a1l5jcIgiDoBR+Dmf2WqjlKzOzH\nmf9n4FMPVB/3T1on0GoYmpubQ17IC3khr19T+gC33kCS9YfrCIIg6C0kYXVyPgdBEDQc48ePR9IC\nsYwfP77b9yd6DEEQLHCk1nK9s9ErdHSt0WMIgiAIchOKIQiCIGhDKIYgCIKgDaEYgiAIgjaEYgiC\nIAjaEIohCIIFnvFNTeWGjDY15c/L+PEMGTKE4cOHM2bMGKZNm8bbb/vXCaZNm8bgwYMZPnw4w4YN\nY/jw4Vx55ZWF349QDEEQLPDMnjMHg9KW2XPm5M6LJG688Ubmzp3LrFmzePDBBzn11FM/2n/MMccw\nd+5c5s2bx9y5c9l5551ruvb2CMUQBEHQYFTGHSy99NJ87nOfY9asWb0qPxRDEARBg/Lcc89x8803\ns8oqq/Sq3FAMQRAEDcYOO+zA8OHDWX755Rk9ejQzZsz4aN+ZZ57JyJEjWWKJJVh66aVLkR+KIQiC\noMG47rrrmDt3LnfccQePP/44r7zyykf7vva1r/Haa6/x+uuv8/LLL5ciPxRDEARBg1HxMWy88cbs\ns88+HHnkkb0qvyG++RwEQRC0z+GHH84KK6zAww8/3Gsyo8cQBMECz7jRoxGUtowbPTp3XqS2E56O\nGjWKvffem5NOOun/7SuLmHY7CIIFjph2O6bdDoIgCLpBKIYgCIKgDaEYgiAIgjaEYgiCIAjaEIoh\n6BM0NeX7eHtT0/h6ZzUI+jylKwZJkyU9LukJSce0s397SQ9JelDSnyRtmPfYYMFhzpzZ5JnH0tMF\nQVALpYarShoAPAFsDrwAzASmmtnjmTRDzOzt9H914Aoz+0SeYzPniHDVfo7Hb+d5xgtOGGLQcyJc\ntb7hqhOBJ81stpl9AFwGTMkmqCiFxGLA/LzHBkEQBMVTtmIYCzybWX8ubWuDpB0kPQb8Bti3O8cG\nQRAExdIQcyWZ2bXAtZI2Ak4GtujuObLT0jY3N9Pc3FxU9oIg6Oc0NY0v1T81evQ4XnrpmVxpx48f\nz8svv8zCCy/MQgstxIQJE9hrr7048MAD2WabbbjrrruQxLvvvoskBg8eDMCee+7Jeeed1+F5W1pa\naGlpyZWHsn0Mk4AZZjY5rR8LmJmd3skxTwHrAh/Le2z4GPo/4WMIiqTa7p6/fPVYYu5yucIKK3Dh\nhRey2WabMW/ePO644w6mT59Oc3MzF1544Ufppk2bxnLLLceJJ57YueQG9DHMBFaWNE7SIGAqcH1V\n5lbK/F8LGGRmr+U5NgiCoD9SqciHDRvGtttuy+WXX87FF1/Mo48+2ivySzUlmdmHkg4BbsWV0AVm\n9pikg3y3nQ/sJGlv4H3gHWCXzo4tM79BEASNyLrrrsuyyy7LXXfdxYQJE0qXV7qPwcx+C6xate3H\nmf9nAGfkPTYIgmBBZMyYMbz22mu9IitGPgdBEPQBnn/+eUaOHNkrskIxBEEQNDgzZ87khRdeYKON\nNuoVeaEYgiAIGpR58+Zxww03sNtuu7HXXnux2mqr9YrchhjHEARBUE9Gjx7HnDnlfTZz9Ohx3Uq/\n3XbbMXDgQAYMGMCECRM46qijOOigg9qkKfMzn/Fpz6BPEOMYgiKJuZLi055BEARBNwjFEARBELQh\nl2KQtKikGE8QBEGwANClYpC0HTAL+G1a/7SkmJoiCIKgn5KnxzAD/zbCGwBmNgtYocQ8BUEQBHUk\nj2L4wMzerNq2YLjzgyAIFkDyjGP4m6TdgYUkrQJMB+4tN1tBEATlMW7cuFLHATQS48Z1bwwF5BjH\nIGkI8E1gy7TpFuBkM3u329JKIsYx9H9iHEMQFEtn4xhigFvQJwjFEATFUtMAN0m3SRqRWV9C0i1F\nZjAIgiBoHPI4n0eZ2RuVFTN7HVi6vCwFQRAE9SSPYpgvafnKiqRxRFRSEARBvyVPVNI3gbsl3QEI\n2Bg4sNRcBUEQBHUjl/NZ0ihgUlq9z8xeKTVX3SScz/2fcD4HQbF05nzO+z2GwcBrKf2EdMI7i8pg\nEARB0Dh0qRgknQ7sCvwNmJ82GxCKIQiCoB+SZ4Db34E1zOy93slS9wlTUv8nTElBUCy1fqjnaWDh\nYrMUBEEQNCp5fAxvA7Mk/R74qNdgZtPzCJA0GfgeroQuMLPTq/bvDhyTVucBXzazh9O+Z4A3cRPW\nB2Y2MY/MIAiCoOfkUQzXp6XbSBoAnAtsDrwAzJR0nZk9nkn2NLCJmb2ZlMj5tEZAzQea06C6IAiC\noBfoUjGY2cU1nH8i8KSZzQaQdBkwBfhIMZjZfZn09wFjM+siPj8aBEHQq+SZK2kVSVdJelTS05Ul\n5/nHAs9m1p+jbcVfzf7AzZl1A26TNFPSATllBkEQBDWQx5R0EXA88F1gM2AaJbTiJVXOvVFm84Zm\n9qKkpXAF8ZiZ3d3e8TNmzPjof3NzM83NzUVnMQiCoM/S0tJCS0tLrrR5wlX/bGZrS3rEzFbPbuvy\n5NIkYIaZTU7rxwLWjgN6DeBqYLKZPdXBuY4H5pnZWe3si3DVfk6EqwZBsdQarvpeciI/KekQSZ8H\nFsspeyawsqRxkgYBU6lyZKcJ+q4G9soqBUlDJC2W/g/FPxT0144ESep0Gd/UlDPLQRAECzZ5egzr\nAo8BI4CTgOHAGWZ2fy4BHml0Nq3hqqdJOgjvOZwv6SfAjsBs3Nn8gZlNlLQCcA3eTBwI/MLMTutA\nRpdtRLnAPFkOGpDoMQRBsdT0BTdJO5vZlV1tqyehGPo/oRiCoFhqNSV9Pee2IOg245uaujQDLigf\nbQ+CRqHDqCRJWwFbA2MlnZPZNRz4b9kZK57BuSqY0aPH8dJLz5SfnQCA2XPm5OwHBEHQW3QWrvoC\n8ACwPfDnzPZ5wFfLzFQ5vEceU8ScOVEFBUGwYNOpj0HSQsAlZrZ772Wp++T1MYSNuvGQ1I0eQzy/\nICiKHvsYzOxDYLkUahoEQRAsAOQZ+fxP4B5J1wNvVTa2N9AsCIIg6PvkUQxPpWUAMKzc7ARBEAT1\npstxDB8lTKOQzew/peaoB4SPoe8SPoYgqA81jWOQ9ElJD+LffP6bpD9LWq3oTAZBEASNQZ4BbucD\nR5jZODMbBxwJ/KTcbAVBEAT1Io9iGGpmt1dWzKwFGFpajoIgCIK6ksf5/LSk44BL0vqe+Oc4gyAI\ngn5Inh7DvsBSwK/TslTaFgRBEPRDuhOVtDgw38zmlZul7hNRSX2XiEoKgvpQa1TSupIeAR4CHpH0\nkKQuv94WBEEQ9E3yfI/hYeArZnZXWt8IOM/M1uiF/OUiegx9l+gxBEF9qPV7DB9WlAKAmd1Nn5x2\nOwiCIMhDnh7D94BFgV/hTbZdgXeBSwHM7C8l57FLosfQd4keQxDUh1o/7Xl7J7vNzD5TS+aKIBRD\n3yUUQxDUh5oUQ18gFEPfJRRDENSHzhRDlwPcJI0A9gbGZ9Ob2fSiMhgEQRA0DnlGPt8E3Ac8Aswv\nNztBEARBvcmjGBYxsyNKz0kQBEHQEOQJV71E0gGSlpE0srLkFSBpsqTHJT0h6Zh29u+eBs09JOlu\nSWvkPTYIgiAonjxRSV8BTgHeoNX7Z2a2YpcnlwYATwCbAy8AM4GpZvZ4Js0k4DEze1PSZGCGmU3K\nc2zmHOF87qOE8zkI6kNNzmf8+wsrm9krPZA9EXjSzGanjFwGTAE+qtzN7L5M+vuAsXmPDYIgCIon\njynpH8DbPTz/WODZzPpztFb87bE/cHMPjw2CIAgKIE+P4S1gVhro9l5lY9HhqpI2A6YBG/Xk+BmZ\n/81pCYIgCJyWlhZaWlpypc3jY9inve1mdnGXJ3f/wQwzm5zWj/VD7fSqdGsAVwOTzeyp7hyb9oWP\noY8SPoYgqA91G/ksaSHg77gD+UXgT8BuZvZYJs3ywO+BvbL+hjzHZtKGYuijhGIIgvrQI+dz+gZD\nh29Ynmm3zexDSYcAt+L+jAvM7DFJB/luOx84DhgJnCdJwAdmNrGjY7uSGQRBENRGhz0GSeM6O7AS\nLdQIRI+h7xI9hiCoDzGJHlGxNCqhGIKgPtT6oZ4gCIJgASIUQxAEQdCGUAzdZHxTE5K6XMY3NdU7\nq0EQBD2iM+fzFWa2SzvRScIjirqMSuotetPH0B2beNi6uyZ8DEFQH3rkfJa0jJm92FF00oIalRSK\noVhCMQRBfYioJEIxNCqhGIKgPtQUlSRpR0lPSnpT0lxJ8yTNLT6bQRAEQSOQZ66kfwDbNfKo4+gx\n9F2ixxAE9aHWcQxzGlkpBEEQBMWSZ9rtByRdDlxL22m3f11aroIgCIK6kUcxDMc/1LNlZpsBoRiC\nIAj6IRGV1E7K8DH0HuFjCIL60NNpt482szMkfZ923siiv+AWBEEQNAadmZIqDucHeiMjQRAEQWMQ\npqR2UoYpqfcIU1IQ1IcehatK2kjS3pn1qyT9IS2fKSOjQRAEQf3pzJR0AnBoZn1V4IvAUOAbwB/K\ny1YQBEFQLzob4DbczB7NrD9pZn82szuBYSXnKwiCIKgTnSmGEdkVM9sxszq6nOwEQRAE9aYzxfC4\npG2qN0raFvh7eVkKgiAI6kln32NYGbgRuBf4S9q8NrABsK2ZPdErOcxBRCX1XSIqKQjqQ4+/xyBp\nMLAHsFra9Dfgl2b2buG5rIFQDH2XUAxBUB/q+qEeSZOB7+FmqwvM7PSq/asCFwFrAd8ws7My+54B\n3gTmAx+Y2cQOZIRi6KOEYgiC+tCjKTEKEjwAOBfYHHgBmCnpOjN7PJPsVTwsdod2TjEfaDaz18vM\nZxAEQdBKnu8x1MJEPMx1tpl9AFwGTMkmMLNXzOzPwH/bOV69kMcgCIIgQ9mV7ljg2cz6c2lbXgy4\nTdJMSQcUmrMgCIKgXTqbXfUROjHqmtkapeSoLRua2YuSlsIVxGNmdnd7CWdk/jenJQiCIHBaWlpo\naWnJlbazcNVx6e9X0u8l6XcPADM7tsuTS5OAGWY2Oa0f64e2dUCnfccD87LO57z7w/ncdwnncxDU\nhx5Nopf8ArOBLczsaDN7JC3H0vZrbp0xE1hZ0jhJg4CpwPWd5TWT6SGSFkv/hyaZf80pNwiCIOgh\neaKSJGlDM7snrWxATt+EmX0o6RDgVlrDVR+TdJDvtvMljca/+TAMmC/pMGACsBRwjSRL+fyFmd3a\n3QsMgiAIukeX4xgkrQ1cCCyeNr0B7Gtmf+n4qN6lMU1Jg4H3ukw3evQ4XnrpmRxn7J+EKSkI6kMh\nA9wkLQ5gZm8WmLdCaEzFUIy8/k7czyCoDz3yMWQOHi3pAuAyM3tT0gRJ+xWeyyAIgqAhyOMr+Blw\nCzAmrT8BHF5WhoIgCIL6kkcxjDKzK/DpKTCz/wIflpqrIAiCoG7kUQxvSVqSZOBNYxMazs8QBEEQ\nFEOecNUj8bEHK0m6Bw8j3bnUXAVBEAR1I1dUkqSBwKp4cMjf04R4DUNEJfU+TU3jmTNndpfpugrH\njfsZBPWh1qikp4D9zexvZvZXM/tA0g2F5zJol/FNTUjqchnf1NSr+XKlYF0ueZRHEASNRR4fwwfA\nZpIuStNaQPdmSA1qYPacOTmqX08XBEFQBHkUw9tmtivwGHCXpOXJ16cPgiAI+iC55koCMLMzJP0F\nn/doZKm5CoIgCOpGnh7Dtyp/zOx3wOfwz3UG/ZC8Po0gCPovnX2P4eNm9riktdrbH5PoNZ68IqJx\nGvn6IiopCIqjs6ikzkxJRwIHAN9pZ58Bnykgb0EQBEGDkXt21UYmegzRY8grLwgCp0c9Bkk7dnZS\nM/t1rRkLgiAIGo/OTEnbdbLPgFAMQRAE/ZAwJbWTspFMH2FKKlZeEAROT53P2RNsA6wGLFLZZmYn\nFpO9IAiCoJHIM1fSj4BdgUPxhtvOwLiS8xUEQRDUiTwD3DYws72B183sBGB94GPlZisIgiCoF3kU\nwzvp921JY/BJ9ZYpL0tBEARBPcnjY7hB0gjgTOAvuAfwp6XmKgiCIKgbXfYYzOwkM3vDzK7GfQsf\nN7Pj8gqQNFnS45KekHRMO/tXlXSvpHclHdGdY4MgCILi6TJcVdJCwDbAeDI9DDM7q8uTSwOAJ4DN\ngReAmcBUM3s8k2YUrnB2wP0YZ+U9NnOOCFclwlXzyAuCwKnpC27Ab4AvAksCwzJLHiYCT5rZ7PQ5\n0MuAKdkEZvaKmf0Z+G93jw2CIAiKJ4+PYVkzW6OH5x8LPJtZfw6v8Ms+dgFkcK7psLv6BnMQBEEe\nxXCzpC3N7NbSc1MDMzL/m9OyYPEeeUwtc+bEtxSCYEGkpaWFlpaWXGnzKIb7gGuSzf8DPjJn2/Ac\nxz4PLJ9ZXzZty0O3jp2R86RBEAQLIs3NzTQ3N3+0fsIJJ3SYNo+P4Sx8UNsQMxtuZsNyKgVwh/HK\nksZJGgRMBa7vJH22OdvdY4MgCIICyNNjeBb4q/Ug1MPMPpR0CP6d6AHABWb2mKSDfLedL2k08ADu\n0J4v6TBggpn9p71ju5uHIAiCoHvkCVf9GbAicDNuyAbyhav2FhGuGvLyyguCwKl1dtV/pmVQWoIg\nCIJ+TKeKIQ1uG2ZmR/VSfoIgCII606nz2cw+BDbspbwEQRAEDUAeU9IsSdcDVwJvVTbGN5+DIAj6\nJ3kUwyLAq8BnMtvim89BEAT9lPjmczspGymKJuQVKy8IAqemSfQkLSvpGkkvp+VqScsWn80gCIKg\nEcgz8vkifMTxmLT8Jm0LgiAI+iF5FMNSZnaRmf03LT8Dlio5X0EQBEGdyKMYXpW0p6SF0rIn7owO\ngiAI+iF5FMO+wC7AS8CLwBeAaWVmKgiCIKgfEZXUTspGiqIJecXKC4LA6dFcSZK+1ck5zcxOqjln\nQRAEQcPR2QC3t9rZNhTYD//+cyiGIAiCfkguU5KkYcBhuFK4AviOmb1cct5yE6akkJdXXhAETo+n\n3ZY0EjgC2AO4GFjLzF4vPotBEARBo9CZj+FMYEfgfGB1M/tPr+UqCIIgqBsdmpIkzce/2PZf2vbh\nhTuf8373uXTClBTy8soLgsDpkSnJzPKMcQiCIAj6GVH5B0EQBG0IxRAEQRC0IRRDEARB0IZQDEEQ\nBEEbSlcMkiZLelzSE5KO6SDNOZKelDRL0pqZ7c9IekjSg5L+VHZegyAIgnzffO4xkgYA5wKbAy8A\nMyVdZ2aPZ9JsBaxkZqtIWg/4ITAp7Z4PNMeguiAIgt6j7B7DROBJM5ttZh8AlwFTqtJMAX4OYGb3\nA4tLGp32qRfyGARBEGQou9IdCzybWX8ubesszfOZNAbcJmmmpANKy2UQBEHwEaWakgpgQzN7UdJS\nuIJ4zMzubi/hjMz/5rQEQRAETktLCy0tLbnSlvqhHkmTgBlmNjmtH4tPp3F6Js2PgNvN7PK0/jiw\nqZnNqTrX8cA8MzurHTkxJUbIyyUvCAKnsykxyjYlzQRWljRO0iBgKnB9VZrrgb3hI0XyhpnNkTRE\n0mJp+1BgS+CvJec3CIJggadUU5KZfSjpEOBWXAldYGaPSTrId9v5ZnaTpK0l/QP/OFDle9KjgWsk\nWcrnL8zs1jLzGwRBEMQ3n9tN2Uimj5BXrLwgCJx6mpKCIAiCPkYohiAIgqANoRiCIAiCNoRiCIIg\nCNoQiiEIgiBoQyiGIAiCoA2hGIIgCII2hGIIgiAI2hCKIQiCIGhDKIYgCIKgDaEYgiAIgjaEYgiC\nIKiBpqbxSOpyaWoaX++s5iYm0WsnZSNN+hbyipUXBEUjib5YNmMSvSAIgiA3oRiCIAiCNoRiCIIG\noD/aqetF3MvaCR9DOykbySYe8oqV19uMb2pi9pw5XScE+uL1NSK9bfMPH0MQBN1i9pw5GHS59FXG\nNzXlap1HVEJnAAAbJklEQVSPb2rqNXlB7USPoZ2UjdTCDXnFyutt4vpSOiisdd5b73p38tRnn130\nGIIgCII8hGIIgiAI2hCKIQjaoa9GtvS2zT/on5SuGCRNlvS4pCckHdNBmnMkPSlplqRPd+fYIOgO\nTSNH5qo458yZTdcuY0vpGoe8zu78kVJFMbjPKVnIV176I6U6nyUNAJ4ANgdeAGYCU83s8UyarYBD\nzGwbSesBZ5vZpDzHZs4RzueQF/K6Ka+3nMEVeV1fX9fO2d52Pjeis7so6ul8ngg8aWazzewD4DJg\nSlWaKcDPAczsfmBxSaNzHhsEQRAUTNmKYSzwbGb9ubQtT5o8xwZB0CP6pmkn6B0G1jsD7dAjo12+\ng3Km6sJumD+DIS/kNaq897pMMWfO7Fw29CKvrzh5xdzLeshrBMpWDM8Dy2fWl03bqtMs106aQTmO\nBejQThYEQRB0n7JNSTOBlSWNkzQImApcX5XmemBvAEmTgDfMbE7OY4MgCIKCKbXHYGYfSjoEuBVX\nQheY2WOSDvLddr6Z3SRpa0n/AN4CpnV2bJn5DYIgCPrJXElBEARBccTI5yAIgqANoRiCYAFAfSUc\npkAWxGsuigVKMfRGQemtwlivQl8PuRWZkob1tuy+jqQ1JQ0yM1sQyoyktSX9ENyJWabs6nOn2Rr6\nBf3mQnIyTtIASYPLOLkkWXLapEiqUqiSM7QsOV3IXaY3ZUr6HPBNSUuXKSvPtrKRNF7SiIJONwO4\noR7KIUUUDrHedWS+BHxC0nehPOVQ9S58QdKSZja/aDn1YoFRDJIOBc4DzgQOL7pCrSooRwA/kLRw\nkTLakfNV4AJJixQtpz0ycr8EfEvSiLIrmvRibwF8H/iNmb1csqxNJO0habuMUuq190TSdsCvgZoU\nb+W5mNkUYB5wRW8qB/kcaDfgSmlPSWNKlle53ueB04DPSjo1bSvtmiUdBnwTWDqzrc+bsBYIxSBp\nW+ALaVkdWMHM3iryAWYqzUOAzwOnmdkHRSuHjJwv49fzdTN7V9KiRcrpCEn7Afvh1/cG0Bs9lqnA\nSWZ2j6RdJP1Y0uFFnTxjqloHuBDYENgTuFbSADOb30uV6STgFGDvFNY9sicVarbxAGBmOwHvA1f1\nhnKQtC6wL7Ar8ENgY2BKmcoh8158DTgY+AOwhaQfVPYXcc2SVpS0aDrfusDuwCbpea0vaUIv95BK\nYYFQDMBiwAV4BWPA9LT9E7UWFkmrSzojs2ll4AhgWGpZ3yRpB0mDapElaTVJP5RUGXuyHHAMsLSk\nrwB/SC2zRcrqOiflMwn4FvDf1Au7WdLXi5aXZE6U9FngGuA4SbcA6wKz8Jd+dBFy0ku+OfB1YH8z\n+zLwReBl4OxKmiJkdcFA4Bbg40nx/QY4TVJz3hNU9Si3kTRF0ipmtgvwDpmeQwn5J5nADgRWMrNH\nzexK4CbgU8DOkkqZ7yyVz5HA9sB0MzsM2AFYVdJpUPszTNd2FG7SXAR4G5gD7CHpXOB04DpJE2uR\n0wj0a8UgaeX09yngf/CX/nNm9n6q1A4Gam3RD8BHaJ+S1l8HTsRNVgLuA7YDPqyxYL4HDAPOkrQQ\nbkv9H+Bk4F3gKuBzwEJFvvSZLrqZ2TvAbcAvgHPxe3cKsJUKtv0nuavirbEbgAOAQ83sa8A9wBJA\nkTbdxfGe3jpp/X3gx/jULKUiaQlJi+HP9D3ga7hSOgZ4GsjtE8sohaPwSmwi8FNJE81sV/y6blEJ\nPjBJqwCbAhcD70k6IeXpOrzcrAEsVKC8bANoMWBu+l8JUngBuBLYX9JJNcpaIfWQr0zn/6qZ/Q34\nEzABuNzMNsEbMZNqkdUQmFm/W/DKehwwG9gNGI7bHb+Lv/xfBP4MfLIGGZOAY9P/NYBfAt9M658A\nlkr/JwO3AyN7KGdd4Mz0f0W851NZH1c5L7AlcCcwqqB7uFLm/77AD4DPpPVVgMXS/88mucNKeI6b\nAn8Els9smwz8FZhS47krgzsXAwal/7sAHwCfTetbAA8CoyrpS7jGHYDrcMW+G+5bGJL2rQ48BGyc\n93oy5eKq9P9buK1/cGb/z4FlS7iWw4HT0//18Knyv5XZv3RJ93AacHz6f1y6Z8ul9S+me7BCDedf\nHLiici3AJsCPgCOBgZl0uwJ/A1Yu4zp7c6l7BkoqKAPS7xTgL3iLfWXcbnwNcD41KIV07k+lCuqo\nzPplwMmZNIcADwOr1yBnBeB+4NS0vhJwEW7iGJiR81AtcqpkLg5cnl6oHYC78Z7B9Xgva2xKNx14\nBFijwGf3SVyJVyruo4CzcDPLsFR5Tk77elRZZ849JT2zK4GJaduueE/kMuAnwOdLLKfr4D3KEcDP\n8Fb1ULyXskkqu91SgEATsAjeq/sJGaWQrq0MBf5JYE1gCNACHJG2r4crvRNreV5dyD4QV96rZrbN\nwHta5+CNw5VqlDEQ95P8AjgmbdsED2Y5DlgU75ndC6xWVnnpzaXuGSihoGwE7A8skta3Bf4O7JbW\nB+DmliJkrZ4q7Uph+RRwKUk5AHsAHytAzni813FGWl8pvfRnp+vZDlilwHs4KL0I56WKa5W0fQqu\nVA9Medoc+ETBz29t4Oq0/A+u9E6nVdkPLEjO54AHgDGp8nykUgmn63wLmFakzHbysCXwZbwX+0dS\nqzZV7mMrlUzeChVYDe9RjsKV6wOZZ7cv3nhYpsD8C29EPISba3dP5eImYO2UZgPg0yXdv4Vws9W6\naT3bK9ogVd49Vgq07YUNAtbHGxFZ5XA2rYpviTKusx5L3TNQYCFRqiQPxbt5e9HaJT8UbwVuU6uM\ndrZ9GrczVgrLGrjT8OiC5YzHW2NZs9LlwCkF3sOlgMXT/41w5fAg8JNMmu2AX+GRSQOKeG7pdzXc\nbFZRAOvjJoKHUyX9PwWXl6Nwc+D2uM/im3gDYpe0f8dUZppLKKsTkuyJuLN5JrBiRu41wNDulhO8\nYXJ1KhsfA85I1/Zd3MRRaGsWGJ5+N0gV5r3A8an8H1/CfVua1NACmnET8TnAN2irFLYAmoool+n/\nCFrNjZNws1/lfd8c+A49NBU36lL3DBRYaFYEFk7/98Tj3vdJ61umSnR8QQXlC7iJZbO0XlEOFbPS\nJ0nmlhrl7J6WHdP6eFw5nJZZL7IF2Ix3/Wfg3eZR6UX4JTAjk27rguVuj0ca/TI9pw1IvbpUwe2J\n914Wr64Mu3tf07MZlf6PBG6slAs8xPFGYHRa35GMiaLA6z0Nny0YPJzze3gPZgu857JdN883PvP/\nOOD3eAt38XTeranBxt7evcSj4m7ATYubZuRsm57hfAr2KaRndxvw0yR7FB5peAawdUqzM/C7oson\nbi69Hg9j3iNtm4SbGk9I64sWXUbqvdQ9AwU9vK/irb2LSS31VKH+LBWSR0gtsgJkHYKbV/YA3gR2\nSNs/BTwJHFaQnENx88J2wH+AA9P2cakSPaGke3kDPiBqo7Q+NFXUlwL/W5CMobT6R9bEnddLJQXw\nIt7j+0wmzRB8+vWP1yh3K+AJYC1ae5iX4+HFE1MFMCmlLcMeXlF2i+JKaJ103celyu5XpF5tXvm0\nmi8vxR3pI/CouA3LKB9V17Eu8BW8l3IzPtakkqYwhYo3WCpmtR/g0Uf7pfXhwLF4o+I2vPdViM8L\nOAi4Azfr/QKPcjos7dso1S9LlnWf67nUPQMFPLz18dZkpXt+Fa3RA6Nxh1shUQLpRb4dd4IekQrh\n+8DUtL8yeK4n5872FFZPhbwi507cnHJ42r88MK6ga6o2R3wJt5veTcY/kl7O75OirWqQNxy4Fm/Z\nKb10a+GK4AE8ousX6Zq3xu3In8S/+d3ja8b9MvdnKv5sgMKluMLoVku9m/LXwBsTG6f1bwD7pv8V\nM0Ul0qtDpVBVToT3DEanSuoivDfSppIu+DoqpsRb8R7OiPTutQAfAj+qur81K9hUVsbhDYrmVEYf\nBHbKpBmDf+Wxx+WznXdhGrAkcFgqsxvjTu3paf8iZZWXei91z0AND3FAqjDmAuelbYNSpXolcHYB\nMrIvYaX1ulQqqHem9SPxbvO2BclZNP2OSpXW3Wl9ryRnWoH3MCt3i+w14Oak+/FW6MZ466mQLnO6\nfzdmK2LgaFpNcbulym2VzP4xNcpcJlWcC6VyUjE7NqVt46vvSYH3ecVUYX8TV3774YrhEdqGBXcq\nu+p5HYzbto/ObNsgVWb/Bv6Fm8oKux68YTQLHxk+De+N75W5v0cB6xUob01gzfR/Bbw3WYke2w33\nPzXjIdPfqFFW9t5un8r9YFzp3kxr+OtV6bn1G0dzu/ej3hkooPDsgUdErJPWB6QCdTEF2Tjx+Oxs\neOi+JJt7kn8+BXSdk5wLaW1B7kqrP2EqbouuyZzSgdyDgMfwVuCfSDHuuEniSTzqpIjoqooJYg28\nhflvWp292+Chhcemyqc5be9RRFBGVkUBjEjXsXUmzUbASZU0JZXPIbgNvOIn+kS6r2fgjZqKiTC3\nIx8339wFfBwfUPkj2o71+DTljFPYBbgos74lroA+lb3nBco7GVc+n07rhwOP0hqFtHNa/zMwoSCZ\nh+C9kYqTeym8h7Rpek9+TI295r6w1D0DPXx40/C4+oNx08Q03MdQKTADKpVrAbIOxG39H89s2xU3\nP1wIPE5qTdQoZ3/cd5E132yH204vBv6RffkLvJcb4FEwlQiTc5OCqCiHiUVWMnjv42+42e+4dG93\nwGPFd8Rt7ZNrOP9YWqPRpuB+gx/hA+PWBF7FRxQfQgED5XLmaSqulCrhqJUGxjeA23Icn23NVsYo\njE7X8Hvcxn85PvCwjB7PJ9PvhqksrkCrwv0eOQbgdVPegMz/n6byWek5fBk3+1V6DsuRggV6KKva\nhDuTqogmfDzPBXgP5VNll5dGWOqegR48yOm44257vHVbcTYfBLxC6jnUcP4BVevfATZI/ytmHuHd\n1wMpKI4fOIHWKKdFM9s3SBVLIT0FWqNzhJsazqquIPEQwJn0MLKqC/kH0Tb8dU/geTLjTLL57MH5\nf4o7DFfHFdxuwE74FBM74lFOJ+At9s/WIquLfCyfZFeu53hax0kMyqT7HalB09nzSv+PwaN/BuOO\n39vT9iXxgIGvU1CDKCNzMt4oWQ33eV0EnIqPvWjGe+trFn3/kuwDcbPwzFQpV5TDl9K73uF9y3n+\nbIjrgFQ2fpfZNqgq/fAyrrMRl7pnoJsPUnhMtvDW0m9xe3FlMNs+1DagZUlSqzy9gAPw1thZVek+\nSw329vYqovTCXVK1bVsKjHqoqmQq92wo3vs6hUzLL1WcNfdQqq8V75JfiLf0KpXmdaTIpIKu81K8\nV3JKZtsEvFdZuCmuHfnb4g2K81PFvykelnp2Jk1lLqjHyNEjwyOqbs6UzzVwc9IY3Ax3eRHPq0rm\nqrg9PVsuFsf9Tz9K+alpbFAnstdKskek9ZPx3lHFrLQvNQSVpHJ/GB6UsH96XoPwwXkHZ9J9KV1r\nzWN2+tJS9wx040FWwid/jPcUrqO19XsgNZgfMjLWT5XWd/HuaqUVcQOtjtE9qMF8VFU5V+K+F8ad\nd7+kdeqLPZOc8SXcy+np/t2O22mb0ov3bdJ8SAXL2wIPKT4UH6R0BW5G2QI3T1wPrF+wzB8k5TAi\ns+08kgmixHI6CY/oqpTXQ3Cn7F142PEXMmmH04EyTPepEqL5xVQWbsjsXwr3j/wBt7MXOgI9yViF\nTJQRrY2Jig+ssEFd/P8GxDK4CWnlzLZr8R5KUeGoO+HRfo/QOrfZdrjZ6Ir0nsyin0xz0a17U+8M\n5HyAI/CPl1RG495L63QF++CtrqKGvp+NT0/8hbQ+ELezP5QKywM9KSikuPnM+jTchHMnbrppxrvr\nv0vXel8ZBRI3p9yPK7zJ6UWbipsJvkua+6VAeevjIX5fwp2ExydlcAo+mdv91BgmSmsDYUPcfLN+\nWr8Ib9VunsrOC5SoGPBIsnuAb7ez7xN4uO83c55rFdwUdjGuwPdK9296Js0S6bw1+7iqZG8I7I2P\nBXoZ2Cqzb0taAy8KaUVXvX9NSSkshIfg7k4KIkl5upUaBq9VyRqa3rVnaPWjDMX9VMfhEYeFOLX7\n2lL3DOR8mEPwSI5K/PAu6SW5Eo8gKKQCxQcLbY23bK8HNqPV3DE45aOns6RmZ2HcBp82QLROXX06\nsFbavwhpaooCrqnaZ7IrqVeS1ifiYwRWwCcaLCziAjd3nE9r5M0iuJnnrEyapvRbk50fdzTPTC/z\n3bSGUV6AO5xPo4TeUEb+ZFK0Gq4IK3MFidYIqWVSHnM584H/xSOXDk7rW+Gt5ulF5z8jcwO8oXUp\nPnX8X/DG0KHpHj8IbF+S7KPSe/dH3AqwLd6zPQ835zxADQNVq5TCOrgDf1iS8xStY0zWp8RItb6w\n1D0DXTzIjWiNL18Nd4JVuujD8QEtRU0zvSIekVMxGR2At07WTgX21BrOvVQq8MJ7IMenF369zP6T\nUyW6QUn3cge8tfdtvIeyKK0t7R9S/Dw6wv0096bKuTIj66J4b6gSF17EAKihuIlxSVzx3Z+tfHHz\nYKGRM1Xy18V7fhNx5XdEqtAqir7SuFgzleFcZRZX1HulynjXtG3tdH27lXAdE/HeSWUQ4Iq4Hf4y\nvCFzDt0cmd0N2Z8Hfpv+XwJcm/5/HDfvfJ2CRlPjJqK7cCf6lXjv5ADcB3Uq8CwlRAD2paXuGejk\n4Q3HWyxP43MTLYfbw79GSdo8yfkercPeD8RbTn+ixsgLvLexBa3OtFNwG+oaaX00HhZX1NiLbOto\nKj446MRUqbyB91C2waOEHqWYkNuKovl4ur5BeK/rl7jJbxw+KPGv1DjJWUbmJ3AT2M9xh/ldJLs0\n3hL8WHX+Ci4zY/E498sz25ZJlc/vSD2HtH0UPRi9nSrGh3FFOyVdY2FzH2XkbIGPXv5GWl84yTuz\no7JVUPkchDcCv4APAryF1qnCa561oEpWRfkJj2C7glbFvTmuCGses9PXl7pnoIMHuXaqKIfgER1n\npJfv9rSMKFDWLvjXmCrrO+Bd1y/hLYnFipKXzv0caTI43AF7NVUtywLkZF+E5fFW9EppfTvcX3I/\nrROEFWZHTZXLJbiz9Di8Bb0VHu1xe5JXiE8h/V8Db+WdiQ+Qq7RoN8VbgGvXIitHXkanyuTPwM6Z\n7cvgPYeawqcz55uMO0lL8T1l5EzBezWV8OFNcfPX0kUohHae35dwM99ueAPs15mKejoe+DGkp7Kr\nZH0Zb3wdT2tUY8Wh/hkKHqDXl5e6Z6CdBzkd75ZXunmL472HlfAW9pPU5mjOOoAXxW2qs/DPfla2\nn5Be9C8X9TJkzj05XUNFOZyMD1gaXISsqhdhelIAj9L2GxU74N3l9SnwWwO43XZ2qkwOx1vyJ+Mt\nwmZcCR5UkKz1KudKSuG7ePTOg3hD4jFqmKYk57V+mmTewHtePyVNqpi2FT2mYGl6YdQt3niYi0//\ncAUF+hSqyuekVDlXegcX4L3LHVL5eZji/Ic7pGtZM70Tf83sOwg3/RX+EaO+utQ9A1UPr7qbd3n6\nXwmPW4SCBpngLZWfpYprLzy0sRLptBtu7y/rU4Rbp4qrYlYqfIbG9CL8nNZ5+c9OlXNl1O1UCppx\nNiNzO1rDG4W3wv6At9AWwZXi9bhJsMetM9zEcV6qvHbHW7lP4b2/tfBxJpVR8GWYj7ZJ5eWIpIg2\nTdsPSBXbTkXL7O2F1qnQv5Z5nrUGCGSVwlbp/b6btrMKHEerv62owaNj8YbQpWl9L9xvciruVP8L\nBX39sL8sFZtw3ZH0ZdK3dfEokm3x1te7kjYDHjCzeQXJ2gnvFeyJh40+h4fJ7Y3P4bMmbpL4exHy\nOsjDDvhLsI4V/BAkjcUjO24zs/0kLYLbbkfgFfPtZvbfAuTIzEzSRPzZzccdlPua2d0pzcX4M30G\nd3xvDDxqZs/3UOZywLt4NMnl+HiTK/E5luYl2c/Wcl1dyF8Vr/x3xq/laNzUcbCZ/VbSwfjEh4+U\nlYfeQtKWuON+upn9usDzfgn3KdyMK6BbgVvM7LlMmoFFlNHM+XbEI5sONLNrJa2B9xSeA64zs0eL\nktUvqLdmSnViV928aymwm4fb9ivRR4NwJ/M5eBTINpQwqKyDfCxW4rl3xOP2K7bigfj0F/9Lmkuo\nIDlTcLPbqbgf4Y94ZfJF3NRyP947OY8av8KGm/5OxmP7N8L9C2fiI3SPxafWKO1D7Li5b1k8imwT\n3FezGB4Q8SaZeP/+suA+o8J6lrgieITWEdzb4j6pfckEQFBOT29b3Dy1c9Hn7m/LwO6pkeJJrdvv\nA3eY2YOSzgW2kXQqXrFNw7/EVkhvIfEoME3STeYthfMltQBmZjcWKKdTzOw/JZ7715LeA06VhJn9\nStLR+HTBbxchQ9IIvNLfDK9ANsG75iNw89GreJjvtZJm4+aeHmNm70g6Ew8PvRD3OS2O+41Ok/RL\nM/tXLTI6QtLn8crrn3gPZSjwSzP7j6RncXv8e2XIridmdlvBpxwD/MrM/pV6BTdI+hC/t+9IusLM\nPrRUkxdJRtb5kuab2dVFy+gvNIQpqbe7ealCOwo3cbTgLdFv4S2+fxcpq95I2gq31x5hZlcWfO6h\neC/kHbyy3svMnpY0Ae+hPF2W8pP0MXx23d2B2WY2UdJCZvZhCbJG4P6oy/EewlfwAII5uInsYHyk\n/IMV81rReegvpPJ4GB4S/ve0bRvcPzXDzF7qhTxsATxlZk+XLauv0hCKAUDStrgN+qSiK7AO5I3B\nzS3b43PYnGBmD5Uttx6U+SJIOgSP3jrMzG6TtCkeOLCzmc0qWl6V7CH4wMdFzezOkmSshzu0R5nZ\nSWnbF/ABV3Pw8MqXzez6MuT3NyQNx01vA/HpQ0bg0XNTo6JuHBpGMUCb1u3hvdXNS5WLzOyt3pDX\n35A0GjcfrYfb3LcFjuxNk1zKR+EtdUkb4PMt/QMPFT0adyx/IGk3PHhgYzN7NXoK+ZG0DO6b2h73\nzZxqZg/XN1dBloZSDBDdvL5IMimtg0/q9ryZzezrFWXqKZyMm+AekXQS3rq9Crg3KYex1sPoqgAk\nDQIws/frnZegLQ2nGIKgEUihmjfhH4I6S9LCwP/gU7NcYma31zWDQVAiA+qdgSBoRMzsVny+/v0k\n7W5mH+DfP3gJn4o6CPot0WMIgk6QtDWuEL5vZj+rc3aCoFcIxRAEXSBpe3yit88Cc8oIiQ2CRiIU\nQxDkQNJS/W2MSxB0RCiGIAiCoA3hfA6CIAjaEIohCIIgaEMohiAIgqANoRiCIAiCNoRiCIIgCNoQ\niiEIgiBow/8Ba8/vhVIWqlEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x28de160f28>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "cols = train.drop(lab, 1).columns.values\n",
    "rf_fi = rf_best.feature_importances_\n",
    "dt_fi = clf_def.feature_importances_\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "width=0.35\n",
    "\n",
    "ax.bar(np.arange(len(cols)), rf_fi, width, color='r', label='RF')\n",
    "ax.bar(np.arange(len(cols))+width, dt_fi, width, color='b', label='DT')\n",
    "\n",
    "ax.set_xticks(np.arange(len(cols)))\n",
    "ax.set_xticklabels(cols, rotation=45)\n",
    "plt.title('Feature Importance from DT and RF')\n",
    "ax.set_ylabel('Normalized Gini Importance')\n",
    "plt.legend(loc=1)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why do Random Forests work?\n",
    "<p>We can show with a little algebra why Random Forests lead to generally better estimation performance.<br><br> Let's say that the variance of a single tree of size $N$ and $p$ features is: $Var(T_b(X))=\\sigma^2$. Similarly the variance of the Random Forest is the variance of the sum of such trees, i.e., $Var(RF(X))=Var(\\frac{1}{B}\\sum\\limits_{b=1}^B \\: T_b(X)).$ Because the invidual trees are trained with overlapping records and features, the individual tree estimates are correlated. Thus, the variance does not factor completely cleanly into a sum of individual tree variances. We'll solve this by using the following relation:<br><br>\n",
    "\n",
    "<center>$Var(\\frac{1}{B}\\sum\\limits_{b=1}^B \\: T_b(X))=\\frac{1}{B^2}\\sum\\limits_{i=1}^B\\sum\\limits_{j=1}^B\\:Cov(T_i(X), T_j(X))$</center>\n",
    "\n",
    "<center>$=\\frac{1}{B^2}\\sum\\limits_{i=1}^B (\\sum\\limits_{i \\neq j}\\:Cov(T_i(X), T_j(X))+Var(T_i(X))$</center><br><br>\n",
    "If $\\rho$ is the correlationb etween $T_i(X)$ and $T_j(X)$, then $Cov(T_i(X), T_j(X))=\\rho\\sigma^2$. So then:<br><br>\n",
    "\n",
    "<center>$Var(\\frac{1}{B}\\sum\\limits_{b=1}^B \\: T_b(X))=\\frac{1}{B^2}\\sum\\limits_{i=1}^B ((B-1)\\rho\\sigma^2+\\sigma^2)=\\frac{1}{B}((B-1)\\rho\\sigma^2+\\sigma^2)=\\rho\\sigma^2+\\frac{1}{B}(1-\\rho)\\sigma^2$</center><br><br>\n",
    "This last quantity shows the two factors that reduce the variance of the Random Forest. 1). The term $\\rho\\sigma^2$ shrinks to $0$ as the correlation between trees reduces to $0$, which is achieved by randomly downsampling the features. 2). The right-hand term shinks asymptotically to $0$ as $B$ increases.\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
